Almost every analytics business case I read has a section near the end about benefits. It carries a large number, written as though the number will arrive on its own once the analysis is signed off. It does not arrive. The distance between a benefit that has been identified and a benefit that has been banked is the most reliable failure in this work, and it is not an analytical failure. Nothing about the model was wrong.
The clearest statement of the alternative I have come across sits in a proposal put to a large European retail banking group by a consultancy selling it a data monetisation capability. Most of that document is what you would expect: a wall of reference engagements from other clients, a catalogue of techniques, a delivery cycle drawn as a loop. What makes it worth reading is the definition it gives its own final stage. That stage is not a report and not a handover. It is defined as embedding the outcome of the analysis into how the business runs, and then continuously monitoring progress against benefit targets.
Two verbs. Embed, and keep watching. Neither is a modelling activity.
The four things named, and what they have in common
The page defining that final stage puts a question to its audience: what matters most to realising benefits. It offers four candidates. A recurring performance conversation among the people accountable for the number. A live tracker of the benefit itself. A sponsor, appointed by name. And a targeting motion that carries the insight to specific customers one at a time.
No answer is printed on the page, so I cannot tell you which one the firm would defend in the room. What I can tell you is what all four share. Not one of them is a technique. The analysis was finished two stages earlier, and the conclusions drawn from it had already been argued over and closed. By the time you reach the last stage, every analytical question is settled. What remains is a calendar and a name against each line on it.
That is an awkward finding for anyone selling analytics, including us. The last stretch of value in a data programme is won by governance habits no data scientist is hired to run.
A tracker built so the miss stays visible
The proposal works that stage through one exhibit, better than most of what I see built for the same purpose today.
Its structure runs in four movements. A funnel and its current conversion ratio. A target for that ratio. A gross benefit potential, expressed as the gap between the two. And then realisation to date, sitting immediately beside the potential rather than on a later page.
The numbers printed inside it are demonstration furniture, and the document says so: the page is stamped as a sample. It shows a baseline conversion of twenty-two percent, a target of thirty-three, a potential of eleven points, and a realisation of seven points against that potential, worth an illustrative five hundred and sixty additional mortgages. The money line is even denominated in a different currency from the rest of the proposal, which is how you can tell it was never anyone's result. None of it is evidence of an outcome and I will not present it as one.
The design is the evidence. Potential was eleven, realisation was seven, and the artefact keeps both numbers in the sponsor's eye at once. Almost every benefits dashboard I have been shown in a bank is drawn the other way, as a line approaching a target, because whoever commissioned it wanted to demonstrate progress. A tracker drawn to display the shortfall makes the gap a standing item on an agenda, and that is the only mechanism I know of that turns an identified benefit into a claimed one.
What the reference engagements show when you read them for cadence
The wall of credentials is meant to prove analytical range. Read instead for what happened after the analysis, it makes a different argument.
One engagement examined the integrated per-client product view described above. Fifteen million euro of build, roughly a million a year to run, and an organisation unable to say whether branches used it or whether the case that funded it had come true. The consultancy counted view events straight out of the system's own audit trail. Over a sixty-seven day window it found more than fourteen million views, and twenty product views accounted for ninety-nine percent of them, while some of the more expensive screens were not opened at all in the period. That is a business case question answered by two months of log data the system had been writing all along. The capability to check had always existed. The habit of checking had not.
A second engagement mined an incident management process costing around twelve million euro a year at the group IT function of a financial institution, reconstructing it from three hundred and fifty thousand calls, one hundred thousand incidents and two point eight million process steps. It identified three hundred thousand fewer steps, more than a ten percent reduction, along with six point eight million hours of throughput time and three point three million euro. Reviewing those results with the client put the savings potential at roughly twenty-five percent.
Three cautions, because they belong to the same lesson. Those figures are identified potential, not realised savings. Throughput time is wall-clock latency, not labour, and must never be added to a euro figure as though it were effort. And the document carries two percentages, above ten and about twenty-five, that it never reconciles. What the engagement did carry, in one line that is easy to read past, is the mechanism: client teams were trained and able to follow up realisation of the benefits themselves. The consultancy left behind a capability to keep score rather than a promise to come back.
A third engagement closed the loop. A bank could not tell whether customers finished a self-service process for changing card limits without falling back to a human. Joining the online process data to the internal record of phone, email and branch contact made the honest definition possible: a journey that ends in a phone call has failed, whatever the funnel says. The bottlenecks were addressed through step order, clearer wording and a more visible entry point, and the success rate rose ten percent. The durable deliverable was not the ten percent. It was a defined method for measuring that process across customers and over time, which is what lets a second improvement be recognised as one.
- 20 views
- Accounted for 99% of more than 14 million views in 67 days
- EUR 15m
- Build cost of the product view whose business case nobody had checked
- 2.8m steps
- Process steps mined across 350,000 calls and 100,000 incidents
- Identified
- Status of the EUR 3.3m and 6.8m hours, not realised
The analysis was finished two stages earlier. Everything on the list that follows it is calendar and accountability, which is exactly why it gets cut first and missed last.
Why this matters more now than it did then
Everything about the analytical half of this work has become cheaper since that proposal was written. Retrieval-augmented generation over internal documents puts a usable answer in front of an operations analyst in an afternoon. Copilots for service staff are being piloted across European banking. A small number of carefully scoped autonomous experiments are running in low-risk back-office corners, watched closely. The modelling that once justified a programme now barely justifies a sprint.
That collapse in cost changes where the risk sits. When the analysis was the expensive part, the benefit case got scrutinised on the way in, because someone had to sign for the spend. Now that a pilot costs almost nothing, nobody scrutinises anything, and organisations accumulate working systems that no one has ever asked to account for themselves. An estate that could carry one unchecked fifteen million euro business case will carry dozens of small ones.
The disciplines that answer this are unglamorous and already understood, and they are where the RealAI Platform work starts. Evaluation sets refreshed rather than frozen at launch. Lineage that survives the join, so a number in a benefit tracker traces back to the systems it came from. MLOps practice that treats a deployed model as something under continuous observation rather than something finished. Data readiness assessed against a specific question, not in general. And above all a named person who has to speak to a benefit line on a recurring date whether it moved or not.
There is a regulatory tailwind too, though it should not be the reason. Europe has agreed the text of an AI law whose obligations will land in stages over the coming years rather than today. The organisations that will find that cheap are the ones already monitoring their deployed systems on a cadence, because much of what will be asked for is evidence that somebody has been watching. Build the habit for commercial reasons and the compliance evidence arrives as a by-product.
The proposal I have been reading was an offer, not an outcome, and I have no way of knowing from it what the bank decided or what followed. The argument inside its final stage stands without any of the credentials attached to it. Value does not fall out of an analysis. It is collected, on a schedule, by someone whose name is on the line.
Drawn from a consultancy's proposal to a large European retail banking group offering to build a data monetisation capability. That document is an offer with a method and a price, not a record of delivered work, and the reference engagements described in it belong to the proposing firm and its other clients rather than to us. Figures marked as samples in the source are treated as samples here. Reading the proposal's final stage as an argument about operating cadence is ours.
“The analysis was finished two stages earlier. Everything on the list that follows it is calendar and accountability, which is exactly why it gets cut first and missed last.”
Get in touch
Put RealAI’s applied-AI team on your hardest data problem.
We help enterprises move from pilots to production: sovereign models, governed data, and agents you can audit. Start with a value-first assessment.