Nobody Can Prove Your AI Program Worked. Fix the Measurement System, Not the Slide.
Photo by Luke Chesser on Unsplash
Every enterprise AI program eventually meets the same room: a CFO, a spreadsheet, and a question nobody can answer cleanly. What did we get for the spend? The uncomfortable truth is that in most cases the answer is unknowable — not because value wasn't created, but because nothing was instrumented to detect it. The widely-cited MIT State of AI in Business research found that the overwhelming majority of generative AI pilots produced no measurable P&L impact, and a 2026 Gartner survey of infrastructure and operations leaders put full ROI success for AI use cases at roughly one in four. Read those numbers carefully and a second story appears underneath the first: these are measurement failures as much as delivery failures. You cannot harvest value you never defined a meter for.
One ROI number is the wrong ambition
Gartner made a point at its 2026 finance conference that enterprise architects should steal outright: AI is not a single investment with a single return, it is a portfolio of structurally different bets. Forcing them into one payback calculation guarantees that the cheap, boring wins subsidise the speculative ones invisibly — and that when the speculative ones fail, the whole programme loses credibility.
Split the portfolio explicitly and measure each tier on its own terms:
- Run-rate efficiency. Deflected tickets, shortened handle times, automated reconciliations. Measure in currency, against a pre-agreed baseline, with a named budget owner who accepts the reduction.
- Process redesign. Value only appears when the process, headcount model, or SLA changes. Measure cycle time and cost-per-transaction — and treat the org change as part of the deliverable, not a follow-on.
- Option bets. Explicitly unprofitable, funded for learning. Measure with kill criteria and time-to-decision, not ROI. A bet closed in nine weeks for $80k is a success.
- Architectural capability. Data contracts, identity, retrieval governance, integration fabric. Measure reuse: how many downstream use cases consumed it without rebuilding.
Unit economics beat business cases
The business case is a one-time document; unit economics are a control system. The most durable metric we deploy with clients is cost per resolved outcome — per closed claim, per answered query, per generated order — tracked weekly against the human baseline. It survives model swaps, vendor renegotiations, and prompt rewrites, and it exposes the failure mode that traditional ROI models miss entirely: inference cost that scales linearly with success. Industry FinOps analyses now put inference at the dominant share of enterprise AI spend, with reported GPU utilisation across enterprise clusters shockingly low. A use case with a beautiful pilot ROI and a bad cost curve is a liability that arrives at scale.
Architecture ROI is measured in avoided work
Enterprise architecture rarely gets credit because its returns are counterfactual. That is not an excuse to stop measuring. Three proxies hold up in front of a finance committee: reuse rate (percentage of new use cases delivered without net-new integration or data plumbing), time-to-first-production for a new use case on the platform versus the first one, and decommission credits — systems, licences, and custom extensions actually retired. In SAP-centric landscapes, the last one is where the real money hides: extension debt removed is a permanent reduction in every future upgrade and every future AI retrieval path. That is the argument we make in SAP modernisation work, and it holds outside SAP too.
Instrument before you build
The practical discipline is unglamorous. Before a use case is funded, agree four things in writing: the baseline and how it is captured, the meter and where it is logged, the owner who accepts the P&L change, and the kill criteria. If the meter requires a manual survey six months later, it does not exist. Our own product work has taught us that measurement built into the workflow reports honestly; measurement bolted on afterwards reports whatever the sponsor needs it to report.
As AI spend moves from discretionary innovation budgets into base IT run-rate, the programmes that survive the next budget cycle will not be the ones with the best demos. They will be the ones that can show a defensible number, weekly, that finance already agreed to count.
Frequently Asked Questions
How do we set a credible baseline when no one measured the process before?
Run a two-to-four week instrumented observation of the current process before any AI is introduced, capturing volume, cycle time, and fully-loaded cost per transaction. Have the process owner sign off on that baseline in writing. An imperfect but agreed baseline is far more useful than a precise one produced after the fact.
What is a realistic timeframe to expect measurable ROI from an AI use case?
Efficiency-tier use cases built on existing data foundations should show a measurable signal within one to two quarters. Process-redesign cases typically take two to four quarters because the value only lands when roles, SLAs, or headcount models change. Option bets should never be judged on ROI — judge them on how quickly they produced a clear go or no-go decision.
Should enterprise architecture have its own budget and ROI targets?
Yes, but measured on platform economics rather than business outcomes it does not control. Fund architecture as a shared capability and hold it accountable for reuse rate, time-to-first-production for new use cases, and decommissioning targets. Attributing business P&L directly to architecture creates disputes that undermine the function.
Comments (0)
No comments yet. Be the first to comment!