Two numbers sat on opposite sides of the same scope. The bank subsidiary had estimated approximately 1,000 person-days for the build. The supplier estimated over 2,500. Nobody reconciled them. The contract absorbed the difference instead, through a clause in section 10.2 of the statement of work: if the work finished early, the savings would be split evenly between the customer and the sourcing partner, and the partner would distribute the resulting revenue pro rata among the project team. The review that followed the project's failure states the rationale for that clause plainly. It existed because the two estimates differed. A commercial mechanism was asked to do the job that a technical conversation had not done.
The challenge
The application under construction was a securities advisory tool, meant to carry the bank's advisory and sales process and to make it demonstrably compliant. Both parties entered in good faith. The supplier had worked inside the client's environment for the previous two years and had succeeded there before. Requirement specifications were understood and accepted, and reflected back in reverse presentations. On paper this was a low-risk repeat.
What was underestimated, in the review's own list, was scale, size, newness of the technology, language and style of management. The delivery team was new to the front-end framework the application was built on. Code quality problems were noticed early, in the first November of the build. A major escalation followed a month later, in December. From the turn of the year onward the status ratings coming from the two sides diverged, which is the visible signature of two organisations working from separate plans.
The commercial picture never became the problem. A financial transactions review sat outside the review's scope, but the client confirmed no dispute on payments or credits, and no commercial overflow on a total engagement of approximately EUR 551K post penalty. Money behaved. Software did not. This is the harder kind of failure to explain to a board, because every control that reports in currency stayed green.
The approach
The review was structured as a 360-degree assessment across three dimensions: delivery approach, project management, and technical quality. It produced thirteen numbered sub-findings, three under delivery approach, four under project management, six technical, each tagged with the party that owned it. Ratings ran on a three-level scale of requires intervention, concerns, and satisfactory. Evidence came from the statement of work and master agreement, meeting notes, the risk register, release planning documents, the project's own reporting pack, and interviews on both sides, alongside a high-level technical review of the 8 May build and a code-difference test against the 18 July version.
Several findings only make sense read together.
The contract had penalties and no credits. The master agreement provided for a malus scheme with no mention of any regime rewarding better-than-expected performance. Accountability, the review notes, was limited to what had been contracted, which made controlling the supplier through the penalty clause close to the only lever available. The penalty was later waived, and the review records that waiving it did not improve collaboration either.
In the case of early completion of the project activities, the project savings shall be split evenly between the Customer and the GSP and the GSP shall distribute any resulting revenue pro-rata among the project team.
There was no evidence of the methodology behind either estimate. The financial package may have been acceptable to both parties, but the review's conclusion is that complexity may simply have been skipped over in the arithmetic. One line item shows what that costs: a system design document describing coding standards, design principles and the rationale for the development approach was estimated at 140 person-days in the statement of work, and the review found no evidence that it existed.
Stacked as units of work rather than plotted as two bars, the disagreement stops being a ratio and starts being a quantity. The contract commenced at 2,576 person-days, so the settlement was not a midpoint between the two positions. One side's arithmetic was adopted whole, and 1,576 person-days of scope that the client had never costed went into the contract with a clause on top of them instead of a shared model underneath. The 140 person-days for the design document sit inside that same stack as the hole they turned out to be, priced into the number both parties signed and missing from the build.
Mitigation went to headcount rather than skill. A senior architect the client believed could recover the project was requested in that December; it took more than two months to place, and by then the review judges it was already too late. Meanwhile the development team was scaled from 20 to 30 on one occasion, which fractured coherence and produced groups building their own modular pieces without seeing the whole. The supplier's project manager was never interviewed by the client team.
Governance was present and did not connect. Escalations were raised regularly at senior level and did not resolve at the execution layer. Reporting was misaligned, with no evidence of any methodology behind how status was flagged. Delivery reviews focused on the immediate issue, deadlines on one side and code on the other, which left cultural awareness and site visits unattended. Penetration testing revealed unacceptable violations in authorization, and the supplier had conducted neither vulnerability assessment nor penetration testing of its own.
The outcome
The code-difference test is the finding that closed the argument. Between the 8 May build and the 18 July version, the review observed no significant improvement in quality and robustness sufficient to allow user acceptance testing to start. Ten additional weeks of effort by a team that had already been reinforced produced nothing the reviewers were willing to call significant on the one axis that mattered.
The recommendation was a shift to a test-and-trust approach: hygiene factors such as experience, depth, skills and quality addressed upfront and monitored actively during delivery rather than asserted at signature, plus a joint lessons-learned workshop between the two organisations. That is a recommendation and a diagnosis, not a recovered project.
For an advisory platform commissioned today, three of these findings have changed shape.
The estimate gap itself is the first. When a coding agent writes a substantial share of an application, person-days stop describing the work. At the time, a 1,000 against 2,500 disagreement encoded two different beliefs about complexity and skill, and the contract converted it into a bet on early completion. The same disagreement today is mostly a disagreement about how much of the scope is agent-tractable and how much needs a human who can say why one instrument is suitable for a given client profile and another is not. That is a question with an answer, and the answer comes from building a slice of the hard part and measuring it, not from a clause. A week spent building that loop, an agent writing against a written spec while a scoring harness grades what comes back, prices the contested part of the scope better than either side's arithmetic did.
The 140 person-days for a design document is the second. Design documentation is now the cheapest artefact in the build and the most valuable input to the machine doing the work. An agentic delivery has to have written down its coding standards, its design principles and its domain rules, because those are the instructions the system runs on, not paperwork produced for an audit file. The failure mode inverts: back then the document was billed and missing, and now a missing one means the agents are inventing their own conventions per module, which is exactly the fragmentation that the 20 to 30 ramp produced by hand.
The third is the code-difference test. Two builds, ten weeks apart, reviewed by experts after the fact, was the best answer available then to a question that should now be continuous. Coverage, authorization violations, design-principle breaches and performance behaviour can all be evaluated automatically on every build, with an agent reading the diff and reporting against a written quality bar. That turns status reporting from an opinion into an observation, which is precisely what was absent when the two sides' ratings diverged at the turn of the year.
Most of the evidence behind that finding was paper, and paper is now machine-readable. The statement of work and master agreement, the meeting notes, the risk register and the release planning pack were read by hand, by people, after the project had already failed. Document and vision models read the same pile on the day each item is signed or filed: section 10.2 surfaces as a commercial hedge against an unreconciled estimate, the design-document line item flags as billed with nothing behind it, and a rendered advisory screen is compared against the accepted requirement specification as an image rather than described in a status call. The same reading applies to what the penetration test produced, so an authorization violation arrives as a ticket in the week it is created instead of a finding in a review.
Wired together these stop being separate checks. The contract reader hands the estimate assumptions to the loop that builds the hard slice, that loop hands its measured result to the agent working against the written design rules, and that agent's diff report feeds the one status view both sides read. It is an autonomous loop carrying the routine share of delivery assurance, running on every build rather than at quarterly review, with the reporting, the code inspection and the security check connected instead of sitting in three different people's inboxes. The reviewers here needed until July to say the May build had not improved on the axis that mattered. A harness says it on the build that fails it, which is the whole distance between a project that stops in November and one that runs to a code-difference test in the following summer.
- 1,000 vs 2,500+
- Person-day estimates
- 140
- Person-days for a missing design document
- EUR 551K
- Engagement value, post penalty
The part that does not automate is the part that failed here. Someone has to notice in November that the quality is wrong, and have the authority in December to stop rather than to add. Agentic delivery makes it cheaper to produce evidence for that decision every single day. It does not make the decision.