OpenAI wants to help companies measure AI value

Many companies have rolled out ChatGPT and Codex, OpenAI's programming tool, to their employees. But most can't clearly answer whether it actually pays off. That's why OpenAI has published a guide and several analytics tools in its administration page, called the Admin Console, which are meant to make it easier to link AI usage to concrete results in the company.

Overview of usage

The first step is to see where the usage is. A tool called Workspace Analytics shows how many employees use AI, how many messages they send and how much it costs, broken down by department. That gives the IT department and procurement an overview of where to focus support and budget.

Next, you can examine what AI is being used for. A tool called Usage Insights sorts conversations into categories such as drafts, summaries or research, based on samples of daily use. It shows whether employees are actually using AI for what the company had hoped, or whether the usage lies elsewhere.

Codex and code quality

For development teams, there's a tool that tracks how much code Codex has contributed, including what share of the finished, approved code AI has helped write. But OpenAI itself stresses that approved code isn't automatically the same as value, and that lines of code say nothing about quality. It has to be seen together with how long it takes to review the code and how many bugs arise.

If the company already has its own systems for analyzing data, the figures can be exported and combined with the company's own key metrics, for example in a report to management. There's also a feature that can automatically put usage and adoption into a finished presentation.

The most important step, though, is linking the usage figures to what actually matters for the business: delivery time, quality, customer satisfaction or costs. OpenAI itself mentions an example from a customer service department, where AI usage is compared with data from customer cases to see whether AI help with writing responses actually shortens response times without more cases being reopened.

Usage is not the same as impact

Here also comes the most important warning, which both OpenAI and independent analysts repeat: that people use the AI tools a lot is not the same as proof that the work has gotten better or faster. Analyst Biswajeet Mahapatra of the research firm Forrester tells the magazine CIO that it isn't enough to measure only how much AI is used, because that shows only activity and not impact. The research firm Gartner makes the same point: companies should also look at concrete business goals such as revenue growth and productivity, rather than just counting usage.

OpenAI itself concludes that the new analytics tools should be seen as a starting point for a conversation within the company, not as a final answer. The tools can show who uses AI and for what. But it's still up to the company's leaders to judge what it's really worth.