Why a Delivery Review Scores Eleven Dimensions and Reports Only Three Verdicts

Every assessment instrument reaches the same fork. You have measured a lot of things, and now you have to say what you found. One road produces a number: a maturity level, an index out of ten, a weighted composite that rolls an entire engagement into 6.4. The other road produces a sentence. The number looks more serious in a pack. It is also the one that spends the next hour of the steering meeting being argued with on the decimal.

I worked on an independent 360 degree review of a failed offshore build for a European banking group's IT services subsidiary. The engagement belonged to the consulting firm I was at, and I led one of its workstreams rather than the whole review. The subsidiary had outsourced the first phase of a customer-facing securities advisory application and hit quality and schedule trouble severe enough to commission an outside post-mortem. I owned the software development process workstream and co-owned the summary of findings page with a colleague on the project management side.

That summary page is the subject here, and I want it as it stood in the working draft rather than as it was finally printed. A draft says more about how an instrument was built than a finished page ever does.

What the page actually carried

Four bucket headers. Project management. Vendor maturity. Technical, subtitled software development process. Delivery approach. Underneath them, eleven named sub-dimensions: project management and planning, governance, collaboration, team location, extensibility, codebase, testing, statement of work, supplier relationship management, contract management, and quality.

Which sub-dimension sat under which bucket is a spatial fact about the exhibit, and I am not going to reconstruct it, for a reason the draft supplies itself. Printed on the page was a working instruction to the two of us: confirm all boxes under correct headings. The people who built the grid had not settled the mapping when that version went round.

The rating legend was written out and complete: three levels, process was good, process could be improved, serious concerns. The headline sentence was not. At draft stage it read overall our 360 degree review found that, and then stopped. The scale was settled before the conclusion was written. That order is the correct one and it is rarer than it should be.

4

Top-level assessment buckets on the summary page

11

Named sub-dimensions carried underneath them

3

Levels in the entire rating legend

0

Numbers, scales or decimals in that legend

Three sentences and no arithmetic

Read the three levels as grammar rather than as a scale and the design decision becomes visible.

Each level is a clause with a subject and a verdict about that subject. Two of the three name the process; the third, serious concerns, names the reader's obligation instead. Not one of them is a position on an axis.

That matters, because a position on an axis invites two moves a sentence does not. The first is averaging. Given eleven scores you will roll them into four, and four into one, and the number that emerges is a fact about your weighting choices rather than about the supplier. There is no arithmetic you can perform on process could be improved. If you want a verdict for a whole bucket, you have to form a judgement and sign it.

The second move is the argument about the decimal. Give a supplier 6.4 out of 10 and what comes back is a case for 7.1: that sub-score was harsh, this one double-counted, the weighting under-values the platform work. Two hours later the room has a revised number and no decision. Give the same supplier process could be improved and there is nothing to negotiate except whether it is true, which is the conversation the review was commissioned to cause.

Three levels also removes the hiding place a five-point scale builds in, where almost everything lands on three and a three usually means the assessor declined to commit. There is no neutral middle in good, could be improved, serious concerns. The middle level already carries a direction of travel.

Process was good / Process could be improved / Serious Concerns

One vocabulary over the contract and the codebase

The eleven sub-dimensions do something that looks careless and is not. They put statement of work, supplier relationship management, contract management and quality on the same page as extensibility, codebase and testing, with governance, collaboration, team location and project planning sitting between the two ends.

Those are not comparable things. A statement of work is a document with clauses in it. A codebase is an artefact you can measure to several decimal places, and the workstream I owned was the one that would have supplied those numbers. The instinct in most assessment design is to give each discipline the metric it deserves: complexity and coverage for the code, clause-compliance counts for the contract, schedule variance for the plan. What that produces is three incommensurable dashboards and an executive holding all three at once.

Grading both in the same three words costs precision and buys transferability. The sourcing lead can read the codebase row. The engineering lead can read the contract management row. Neither needs the other's units, and the page can be read straight down by someone who owns none of it, which is the entire point of an independent review. That is a different reading of this engagement from the one I took of its findings summary in Why Your Supplier Health Check Stops One Layer Above the Code, which looked at who was named in each finding rather than at how the dimensions were scored.

The cost is real. A measured coverage percentage and a missing contract clause both come out as one of three phrases, and the phrase carries no magnitude. Which is why a scorecard is a routing layer and not the report. The magnitudes live in the sections underneath, and the scorecard sends the reader to the one that holds them.

The scale was settled before the taxonomy

The most useful thing on the draft is the sequencing it records by accident.

The legend was written out in full: three levels, fixed wording, printed. The taxonomy was not. The authors were still asking themselves which box belonged under which heading and would resolve it in a working session a few days later. The headline sentence was an ellipsis.

Most assessment design runs the other way round. Teams spend weeks agreeing the dimension tree, because the tree is where stakeholders have opinions, then choose the output format in the last hour before the pack goes out. That is how a review ends up reporting in nine colours, or in a percentage nobody can source.

The tree is negotiable. Dimensions get merged, split and renamed right up to the deadline, and it rarely changes what the review concluded. The output vocabulary is not, because it decides what the reader can do with any of it. Fix the sentence set first. Argue about the boxes afterwards.

What inverts once scoring gets cheap

Eleven sub-dimensions was expensive at the time. Between them they meant stakeholder interviews, document reads and a sampled review of the code, and eleven was roughly what a small team could evidence in the weeks the review ran. Breadth was the binding constraint, and the coarse output was in part a consequence of it.

That constraint has gone. An agent with read access to a repository, a defect tracker, a contract set and a meeting record can score hundreds of dimensions nightly, re-score on every merge, and keep the history. Eleven becomes a thousand at a marginal cost close to nothing.

What has not become cheaper is the capacity of a delivery director to act on what comes back. That sits at roughly the same handful of decisions a quarter it has always been. So the constraint moves. The scarce resource is no longer measurement, it is the output layer, and the output layer is the part teams are currently least careful about, because a machine is happy to emit anything you ask it for.

The failure mode is already visible in AI-generated assurance dashboards. Hundreds of dimensions, each with a score to one decimal place, a trend arrow and a confidence band. It reads as rigour. It cannot be escalated, because nothing in it says what to do, and a supplier handed it will spend the meeting on the twelve dimensions where the model was most obviously wrong.

Machine breadth underneath, human coarseness on top. Score everything continuously and report it in three sentences. The agent finds the dimensions that changed level and carries the evidence for the change. The scorecard stays small enough that a person can hold it and defend it in a room. An instrument that emits more distinctions than its reader can act on has not become more precise. It has moved the judgement from the assessor, who was accountable for it, to the reader, who is not.

Drawn from the summary of findings page of a working draft of an independent 360 degree delivery review, commissioned by a European banking group's IT services subsidiary after an offshored application build failed. The engagement is a consulting firm's, and I owned the software development process workstream and co-owned this page. The four buckets, the eleven sub-dimension names, the three-level legend and the authors' own note to confirm the box headings are the page's. No verdict is reported for any dimension, because the verdicts were carried as colour rather than as text, and the mapping of sub-dimension to bucket is deliberately not reconstructed here. The review produced findings and recommendations, not a repaired application. Reading the coarse output layer as a deliberate design constraint, and the inversion that follows once continuous scoring is cheap, is ours.