AI Success Depends on Business Outcomes, Not Usage Metrics

Companies should judge artificial intelligence by whether it improves business outcomes, not by how many tokens employees use or how many people have access to a tool, argues Matt Finlayson in a TechRadar Pro Perspectives article.

Finlayson describes internal leaderboards that ranked employees by token use, a trend dubbed “tokenmaxxing,” and a later push to minimize token use. He says both approaches can distract from the goal: delivering more products at higher quality with the same or fewer people. He also warns that counts such as GPU utilization and AI features shipped per quarter can reward activity without showing whether the work helps.

As an example, Finlayson says 95% adoption of GitHub Copilot may indicate licenses were distributed, not that developers are using it effectively. If cycle time has not fallen by 30% or rework has not decreased, he argues, adoption alone does not demonstrate improved outcomes. Faster code generation can also leave review as a bottleneck or increase bugs if AI-generated errors are approved without sufficient scrutiny.

He recommends starting with a specific business outcome and measuring it before and after AI enters a workflow. Engineering teams could use DORA or SPACE as measurement frameworks, while other teams might track targets such as response time or forecast accuracy. In one example, a support team reduced first-response time from nine hours to three hours after agents began drafting replies with an AI assistant; reopen rates did not change.

Finlayson says leaders should define the outcome they want to improve before choosing an AI metric. The views in the article are his and are not necessarily those of TechRadar Pro or Future plc.