Why One AI Maturity Score Hides the Gap Between Model Quality and Workflow Integration

Most capability assessments arrive as one number. A digital maturity score, a data maturity score, and now an AI maturity score, each rolled up so it can sit on one chart beside every other business unit. The roll-up is what makes it presentable, and it is also what makes it close to useless, because a composite is an average, and averaging capabilities that fail independently deletes the one fact a delivery manager can act on.

A decade ago I worked on the integrating platform blueprint for a global staffing and HR services group. The document had to reconcile two approved business propositions onto one platform, and most of it reads like any target-state deck of that period. One formatting decision in it has aged better than almost anything else on the page.

The blueprint did put a headline level on the experience target for the professional servicing model, and that headline is the part that failed. The slide setting the targets calls it relationship-driven, the fourth rung of five. Two later slides, one of them the grid for that same proposition, say experience-driven in their titles, which is the fifth. Three slides, two answers.

Underneath the headline the same document is exact. The target experience state is written as two capability tracks laddered in parallel across the same five steps: one for the content the group produced, one for the channels it reached people through. Roughly thirty discrete named behaviours sit across the two, each a thing somebody is either doing or not doing. The score moved between slides. The behaviour inventory did not.

The two tracks fail independently, and that is the whole point

The content ambitions are editorial and production work. Building enough authority on talent and career development that people treat the group as a source rather than a vendor. Producing in multimedia formats, video and infographic, instead of text alone. Curating third-party material and attaching the group's own analysis to it. Syndicating out to affiliates, social networks and third parties. Inviting customers and candidates to contribute as guest bloggers. Packaging content for the individual professional so it can be distributed one to one across every channel.

The channel ambitions are plumbing and operations. Analytics that track engagement well enough to prioritise what gets built next. Social analytics that pick up sentiment rather than volume. Customer data shared and acted on in every channel rather than only in the one that captured it. Physical branch environments adapted to digital. Media planning and execution run across channels instead of per channel. A 360 degree view of talent, used to tailor engagement. An omnichannel workflow operation with automation behind it.

Nothing on the first list produces anything on the second. A group can hire editors, commission video, build a taxonomy and become genuinely good at content while its channels stay as the document recorded them at the outset, unintegrated and fragmented, media planning siloed, journeys breaking at every crossing point. That combination is common, expensive, and invisible to any instrument that reports one number.

Two organisations can post the same composite score and need opposite programmes. A number that cannot tell them apart is not a measurement.

What the average does to an investment decision

Grade both tracks honestly, then roll them into one figure. Strong content, weak channels, and the result lands somewhere in the middle.

Now the mirror image: integrated channels, engagement tracked across touchpoints, an operations team that can act on customer data wherever it appears, and almost nothing worth putting through any of it. Same middling figure.

The two need opposite programmes. One should stop commissioning and spend a year on integration. The other should stop integrating and hire editors. A composite score cannot tell you which one you are holding, and the plan written from it is built on whichever of the two its author privately assumed.

The second cost is about accountability rather than arithmetic. A track-level record puts a weak capability in front of the person accountable for it, as a named behaviour that is not happening. A rolled-up score spreads that weakness thinly across everyone who contributed, and a finding owned by everyone is owned by no one.

The same divergence, wearing new nouns

Grade model capability and workflow integration separately and the shape of most enterprise AI estates becomes clear inside an hour.

On the reasoning track, organisations are usually further along than they claim. There is gateway access to frontier models, retrieval over part of the document estate, a prompt library someone maintains, a small evaluation set, and people who can tell a good output from a bad one. Handed the right context by a human, the model does the task.

On the integration track, most are close to where that staffing group's channel row started. The agent cannot read the system of record unless a person pastes into it. It has no identity of its own, so what it writes goes in under a shared service account, or does not go in at all. There is no exception path, so a case it cannot finish stops rather than routes. There is no log a compliance function would accept as evidence of what it did, on whose behalf, under which policy.

Averaged, those two produce a comfortable middling readiness score and a plan that buys more model. Kept apart, they produce the plan the evidence supports: the next two quarters are identity, permissions, write paths, exception routing and audit, and no additional model capability moves the completion rate until that work lands.

This is the modern form of brilliant content on broken channels. Impressive demonstrations, a completion rate close to zero, and a programme review that keeps asking whether a better model would fix it. The reasoning was never the binding constraint.

The same blueprint shows how easily the discipline gets dropped. Its data capability was laddered as a single row across the same five levels, with no second track beside it, and the drafting gives that away: the row labels belonging to the experience grid were left sitting on the data slides, where there is only one ladder to label. One capability got two tracks. The next one in the same document got one.

A published case study on the same blueprint reads that data grid rather than the experience grid, and finds a career agent written into a cell in six words, years before anything could build it: One Cell in a Data Maturity Grid Specified a Career Agent a Decade Early. That piece is about what the cells said. This one is about why there were two rows.

2

Capability tracks written in parallel for one proposition

~30

Discrete named behaviours across the two tracks

5

Steps in each track, identically spaced

3

Slides naming the headline level, giving two answers

What to put on the page instead of a score

Four things, none of which need a new instrument.

Split the assessment before you grade anything, not after. A split applied to a finished composite is arithmetic and changes nothing. A split applied beforehand changes which questions get asked, because the questions that interrogate reasoning quality and the ones that interrogate integration barely overlap. One is about outputs, the other about permissions and failure paths.

Write each track as behaviours that someone is either doing or not doing. "The agent writes to the case record on its own identity, and that write appears in the audit log with the policy it ran under" can be checked in ten minutes by one person with access. "Integration maturity, level three" cannot be checked at all, which is why nobody in the room argues with it.

Publish the divergence as the finding. The gap between the two tracks is the deliverable, not a footnote under a headline number. If the two come back close together, either the estate is unusually even or the assessment is not discriminating, and in my experience the second explains it more often.

Refuse the roll-up when it is asked for, and it will be asked for by whoever wants five business units on one slide. Offer two numbers and one sentence about the distance between them. A steering group that cannot hold two numbers will not hold the programme either.

The blueprint behind this piece was a working draft, and it recorded a target state rather than a delivered one. Nothing on either track is reported as achieved. What survives is the format decision. Somebody wrote two rows of behaviour underneath a headline level that would have presented far better on its own, and a decade later the headline is the part that contradicts itself while the two rows are the part a reader can still use.

Source: the target-state experience pages of an integrating platform blueprint I worked on for a global staffing and HR services group, a version 0.9 working draft rather than a record of a built system. The two capability tracks, the named behaviours on each, the current-state note about fragmented channels and siloed media planning, and the disagreement between the target level set on the targets slide and the one carried in two later slide titles are all as written in the source. The reading of the two-track format as the finding, and the mapping onto reasoning quality against workflow integration, are ours.