Two Chains of Evidence

Every page of that assessment carried the same footnote, eighteen times: this is a point-in-time assessment of a build dated to one particular day. The document was warning the reader about its own shelf life before it said anything else. By the time the findings were written up, roughly seven months had passed since the build it described. The most repeated sentence in the deliverable was a disclaimer.

That footnote is the honest starting point for anything written about code review, and it is the reason the exercise has to produce two chains of evidence rather than one.

The engagement was an independent review of a new advisory application at a European retail and private banking group. .NET on the server, a JavaScript framework on the front end, built by an external development vendor against a signed agreement. I led the software development process workstream. The instrument was a grid: eighteen pages, one per quality aspect, each split into narrower topics, and each row carrying two columns. One pointed at the defect tracker. The other pointed at an article ID or a section of the governing document.

Two columns, two chains. What the code does, and what the paperwork says it should do. Almost nobody reads an assessment grid that way, and it is the only way worth reading it.

The chain that can defend itself

Start with the case where both chains fire and disagree.

Compliance with an agreed set of object-oriented design principles was written into the project agreement with the development vendor, and validated through an internal software quality process that required any application built under it to comply. That is the document chain, and it is unambiguous. The code chain said something else. Hundreds of classes, each large and responsible for many actions. Many controllers at around two thousand lines, which the review flatly recorded as making single-responsibility compliance impossible. Dependencies running between service and controller, breaking the layering pattern the design claimed to follow. A cyclic dependency large enough that the layers could not be pulled apart at all, leaving a monolith wearing the vocabulary of a layered architecture.

The review's own phrasing was that this application had more violations than a typical application. Not a novel category of defect. A familiar one, at unusual volume.

Now notice what the reconciliation actually bought. Without the document column, all of that is a reviewer's opinion about class size, and any competent development lead can argue it down as taste. Without the code column, the agreement is a promise nobody tested. Put the two beside each other and the finding writes itself, because the standard was not the reviewer's, it was the client's own, agreed in advance and signed by both parties. The severity is no longer a matter of judgment. It is a matter of arithmetic against a clause.

This is the part practitioners tend to get backwards. The document chain is not the weaker evidence because it is prose. It is the weaker evidence because it cannot generate anything new. Code can be re-run, re-measured, re-instrumented, and it will answer the same question again next Tuesday with a fresh number. A clause can only be re-read. When the two disagree, one party to the argument is able to produce new evidence in its own defence and the other is not.

Where only one chain fires

The more interesting rows were the asymmetric ones, and there were two kinds.

First, code with no clause. The testing page is the clearest example in the deck and carries its hardest numbers: coverage on the server side measured at approximately fourteen percent using the platform's own coverage tooling, front-end coverage that could only be estimated and was most probably below ten, and the observation that most of the tests that did exist were not very useful, because they asserted against null and length rather than against content. That page references no defect tickets. It could not. A tracker records things that were found. It has no way to evidence a thing that was never written. The only chain available for an absence is direct measurement.

Second, clause with no code. One page traced responsiveness on mobile devices not to a tracker at all but to a section of the statement of work. A contracted requirement, with the build's performance against it recorded on the document side because that is where the obligation lived. A defect tracker cannot evidence a broken promise either, for the same structural reason it cannot evidence an absence: nobody logs a ticket against a commitment they do not know they made.

So the honest count is three chains, not two. Tracker evidence, direct measurement, contractual obligation. The two-column grid was already carrying the third quietly, and I did not name it at the time.

35

Distinct tracker findings, across 12 of 18 assessed aspects

43%

Share of all findings sitting in design alone, on two pages

~14%

Measured server-side test coverage, with front-end estimated below 10

13 of 18

Pages carrying an unfilled findings placeholder

Where the document is the thing that is wrong

Two chains means either one can be the defective party, and the temptation is to assume the code is always the guilty one. In this review it ran both ways.

The clearest case sits in the shape of the instrument itself. The grid presents eighteen aspects as co-equal, one page each, an even distribution of attention across quality dimensions. The code says otherwise. Design took forty-three percent of all findings on two pages. Architecture, which the design page shows to be the deepest problem in the build, got one page, one row and one tracker reference, with its entire substance stranded twelve pages earlier. Security, in an assessment of a bank's advisory platform, came back with four findings out of thirty-five, and two of its four aspects recorded nothing whatsoever. I will not explain why security was thin, because the deck does not say and a separate workstream may well have covered it. The distribution is arithmetic on my own tables. The explanation is not available and should not be supplied.

A fixed grid guarantees coverage and destroys weighting. That is the document being wrong about the system it describes, and only the code chain can catch it.

Then there are the blanks. Four aspects came back with nothing, and the document never distinguished between examined and clean, and not examined. One page did the honest thing and wrote not applicable against deployment, which is an affirmative scoping decision a reader can act on. Thirteen pages carried the template word for findings with nothing under it. A blank that cannot say which kind of blank it is has failed at the only job a document has.

The best small example is a single sentence on the concurrency page, describing thread behaviour, which as written says the opposite of what the surrounding paragraph argues. Almost certainly a dropped negation. Nobody reading the document alone would catch it, because it is internally fluent. Only someone holding the code chain beside it would notice that the prose and the build had come apart in a single word.

What actually got repaired

One repair is recorded anywhere in those eighteen pages. Front-end race conditions and some server-side static variables, described in the same sentence as easily identifiable, fixed a few months after the assessed build. Every structural finding, the cyclic dependency, the two-thousand-line controllers, the missing front-end data model with payloads running up to a hundred kilobytes across the wire, the coverage figures, carries no remediation note at all. I am not claiming those were never fixed. I am claiming the record shows no fix.

Remediation followed detectability, not severity. That is the most durable thing in the deck, and it held because detecting a race condition was cheap and detecting a broken layering contract cost a senior reviewer a week.

That economics is what has changed. Static and structural analysis now runs on every commit rather than once against a seven-month-old build. Clause-to-query mapping is work an agent loop can carry, under graded autonomy, with a human approving the mapping rather than performing the grep. Retrieval over the contract set puts the obligation and the code path in the same context window. Under the EU AI Act and under any serious model risk management regime, that reconciliation is no longer an annual deliverable with a disclaimer footnote. It is a control that has to run continuously, with an evaluation set behind it and an audit trail in front. RealAI's Agentic OS is built for exactly that loop: the specification on one side, the running system on the other, and an agent whose job is to keep reporting where the two have come apart.

Code can be re-run. A document can only be re-read. When the two disagree, only one of them is able to produce fresh evidence in its own defence, and it is never the document.

The footnote on every page was right. Any assessment of a build is out of date the moment it is written. The answer was never a better document. It was to stop treating the reconciliation as an event.