Every lending operation I review reports one number for speed. Time to decision, or time to cash, averaged across a month and rounded to a day. By the time it reaches the board pack it has been stripped of the one property that would let anybody act on it.
I have been carrying a sentence about this for years. It comes from an appendix page in a data proposal put to a large European retail banking group.
The page was addressed to the operations side of the house, marked for discussion, and carried no figures at all. It said that collecting data in all steps of the processes gives insights in "lead times (and distribution), used resources and quality". Somebody wrote those two bracketed words because they knew the noun on its own would be read as an average, and an average was not what they meant.
The average describes a process nobody runs
The clearest demonstration sits in a case recorded later in the same file, at a group IT function inside a financial institution. The process was incident management, costing roughly EUR 12 million a year, with incidents described as bouncing back and forth between departments and suppliers. Process mining was run end to end across the service management chain, and the volumes are stated plainly: 350,000 calls, producing 100,000 incidents, producing 2,800,000 process steps.
Divide the last two and you get twenty-eight steps per incident. That division is mine, not the document's, and it is worth doing for how absurd the answer is. No target operating model was ever drawn with twenty-eight boxes in it. Twenty-eight is what you get when most incidents close in a handful of steps, a minority go round and round, and somebody averages the two populations together.
That is what an average does to an operational measure. It converts a shape into a scalar, and a scalar admits one remedy, which is more capacity. Hire another underwriter, add a shift to the servicing centre, move the queue offshore. In lending that is usually wrong, because the file that sat for a season did not take a season of work. It spent that time waiting, in a queue nobody could see, for a document nobody chased.
The improvement identified was 300,000 process steps, more than ten percent of the total. Against the average that sounds like a trim you would get from tidying. Against the distribution it is not, because you cannot take a tenth off a file that only ever had a handful of steps. The whole reduction has to come from the cases that loop, so it is only findable if you can see which cases loop.
Two honesties belong with those figures: they are identified potential rather than realised benefit, and the hours reported are throughput time, wall clock rather than labour, which is the difference between a queue emptying and a person being freed.
The tail lives between the departments, not inside them
The phrase the document uses is bouncing back and forth between departments and suppliers. That is not a performance failure inside any team. It is a property of the handovers, and handovers are the one thing no team's own reporting can see, because each department measures from the moment work reached its desk to the moment it left. Every one of those local averages can look defensible while the end-to-end time is dreadful, since the waiting happens in the gaps and nobody owns a gap. Which is why the proposal put the customer's perspective ahead of any technique, and also why this work stalls politically: a distribution attributes delay to a route through the organisation rather than to a team, and routes have no owner on most org charts.
Variance between units is the shallow version of the same idea
A third case in the same file belongs beside this one. An insurer setting out to reduce product and process complexity had no visibility on how well it served customers across channels. Transaction data assessed across its branch estate returned dispersion: cross-sell ratios differed sharply between local branches, and product lead times per local branch varied by as much as 14 days compared with the average. The spread was offered as a way to distil the target from the firm's own best-performing branch rather than design a new one. Whether the client then did that, the document does not say.
Taking the target from your own best-performing unit is the right instinct, and it is still a set of averages. Ranking units tells you which branch, not which file. A branch with a respectable mean can be the one holding your worst cases, and a branch with a poor mean can be uniformly mediocre, which is a different problem with a different fix. Both cuts are needed. Most operations packs carry the first.
Why this used to be expensive and is not any more
Obtaining a distribution used to mean a study: interview the participants, draw the process on a wall, sample files by hand, argue about whether the drawing was fair. That produced the process as documented, at considerable cost, and the documented process is exactly the thing that is not true.
Process mining reads what the estate already writes. A workflow system, a servicing platform or a case management tool stamps a time and an actor onto every transition as a by-product of routing the work, in the same way a telephone switch counts before anyone asks it to.
Which relocates the data readiness question. It is not volume. It is three properties of the log: a case identifier that survives every handover, so a file moving from origination into servicing is still recognisably the same file; a timestamp somebody will defend; an actor on each event. Estates fail on the first far more often than the other two, and that is a lineage problem rather than an analytics one. Where the identifier breaks, the distribution truncates at the system boundary and you are back to an average, wearing a better chart.
What changes when models enter the same process
Take evaluation first. Put a copilot into a lending back office to draft a credit summary or pre-fill a decision memo, and the honest question is what it does on the files that go wrong. An evaluation set sampled from typical files is a set of easy files. It will score beautifully and tell you nothing, because in a lending book the tail is where the losses are. A distribution is how you find those cases.
Then anything that learns the process from the process. Early and carefully scoped autonomous experiments in an operations chain are grounded on the log of the work as it is done today, and a routing model fitted to a chain that bounces will learn to bounce, faster and cheaper. Retrieval over the process documentation returns the process as written, which is the account that was already wrong. The event log is the only description of the process as run.
- 2,800,000
- Process steps mined across 350,000 calls and 100,000 incidents
- 28 steps
- Average per incident, our division of the document's own two figures
- 300,000 steps
- Identified reduction, stated as more than 10 percent
- 14 days
- Widest branch lead-time gap from the average, one firm's own branches
Nobody designs a twenty-eight-step incident. That figure is not a description of the work, it is the arithmetic shadow of a distribution with a long tail, and the tail is the only part worth touching.
What to instrument, and in what order
One case identifier that survives every handover, declared and tested across the systems a file passes through. A timestamp and an actor on every transition, recorded from where the customer's clock starts rather than where a team picks the work up. Percentiles in place of the mean: median, ninetieth, and a count of files that broke the promise. One named owner for the route rather than for each stage of it.
The second idea on that same appendix page reads like an afterthought and is not. Let customers maintain their own information in a digital environment, so errors surface early instead of arriving as an exception three weeks into a file. Much of a long tail in lending is not slow work. It is rework caused by something wrong on arrival, and the cheapest place to catch that is while the customer still has the document in hand.
Do those and you have built nothing yet. What you have is a process whose shape you can see, and a defensible view of which part of it a model could carry. That is the opening block of a RealAI Platform engagement.
Sourcing: the bracketed line comes from an appendix page of opportunities, marked for discussion, in a data proposal put to a large European retail banking group. That proposal produced recommendations rather than results, and no idea on the page carried a figure. The process-mining volumes and savings come from a customer case in the same document, at a group IT function inside a financial institution, stated as potential rather than realised; that case also reports a savings potential of approximately twenty-five percent beside its more than ten percent step reduction, and bridges neither. The 14-day branch spread comes from a third case, at an insurer. Reading all three as one argument about averages is ours.
“Nobody designs a twenty-eight-step incident. That figure is not a description of the work, it is the arithmetic shadow of a distribution with a long tail, and the tail is the only part worth touching.”
Get in touch
Put RealAI’s applied-AI team on your hardest data problem.
We help enterprises move from pilots to production: sovereign models, governed data, and agents you can audit. Start with a value-first assessment.