Ten Questions Before You Outsource

Every failed outsourcing engagement ends with a list of questions. What makes this particular list worth publishing is that the client wrote it, wrote it down, and handed it over before any reviewer touched a document.

The client was a European banking group's IT services subsidiary. It had outsourced the build of a customer-facing application to an offshore development partner it had worked with before. Delivery problems were visible almost immediately. Code quality was escalated formally into the steering committee inside the first quarter. A second escalation followed on data context. Neither resolved. The subsidiary granted a grace period of ten weeks for the supplier to get the application stable enough to enter user acceptance testing, and at the end of it the application still crashed and still behaved erratically. The statement of work was terminated and the build taken back in-house.

Then the subsidiary commissioned an independent review, and brought ten questions with it. I led that review. What it produced was findings and recommendations, not a fixed system and not a recovered programme. That distinction matters, because the argument here is about when a question gets asked.

One yes, one both ways, eight no

The answer sheet is the shortest page in the report and the only one people remember. Ten questions, ten verdicts. One yes. One recorded as yes and no at once, because project management was competent in some layers and absent in others. Eight flat negatives.

Look at where the yes lands. The requirements specification met accepted international practice, was written well, and was read back to the client by the supplier in its own words. The opening artefact was fine. Everything the ten questions asked about the distance between that artefact and a working application failed.

That is the shape of most sourcing failures I have reviewed. The specification is rarely the problem. Organisations are good at writing down what they want, much worse at writing down how they will know they got it, and worse again at making that second thing contractual.

Read backwards, the list is a specification

The move this piece exists to make starts there. Every one of those ten questions is answerable in advance. Asked at the start, each is a design instruction about what evidence the engagement will produce. Asked at the end, each is a verdict you cannot appeal.

Take the question of whether the vendor delivered the output specified in the contract. Asked at the end, it is forensic: assemble the contract, the addenda, the release notes and the builds, and adjudicate. Asked at the start, it is much smaller. What will acceptance evidence look like, who produces it, where does it live, and which artefact settles a dispute about whether a sprint delivered?

Take the maintainability question. At the end it is an opinion, contested by definition, because the supplier is being asked to agree its own work is a liability. At the start it is a list: the design principles the codebase will follow, the coverage floor, the review gate that has to pass before a merge, the person on the client side who signs that it passed. None of those are expensive. All of them are boring. They get skipped because they feel like paperwork at the moment everybody is excited about starting.

Take the question about whether project management was sound at every level. Governance existed and was used. Escalations went up. Steering committees met. The gap the review found sat between the senior level where escalations were raised and the execution level where nothing changed as a result. A senior architect the client trusted to turn the build around was requested, and it took more than two months for that person to appear. By then the codebase had set. A pre-mortem version of that question is not "will we have governance" but "what is the maximum time between an escalation and a change in what the delivery team actually does, and what happens contractually when we exceed it".

The evidence that had already been paid for

The single most instructive finding in the review is also the least dramatic. There was no system design document. Nothing set out the coding standards, the design principles or the reasoning behind the development approach that was chosen. That is not an oversight of scope. That deliverable was itemised in the statement of work at 140 person-days. It was specified, priced and never produced, and nobody noticed until a review went looking for it a year later.

The estimation finding rhymes. The client put the work at roughly 1,000 person-days by a method it could name. The supplier came back above 2,500. A profit-sharing clause went into the statement of work partly to bridge that gap. What the review could not find was evidence that either side had used a repeatable estimation method it could show its working for. Two numbers, more than a factor of two apart, and no shared basis on which to argue about the difference.

Reporting had the same hole in a different place. Both parties produced status flags. Neither had agreed what the flags meant. Ratings diverged partway through the engagement, and the divergence was a symptom: two organisations were colouring the same programme differently because nobody had defined the colours. The recommendation that follows is embarrassingly simple, which is to agree pass and fail definitions and status definitions jointly, before the first sprint rather than after the first argument.

One pattern runs under each of them. In every case, an artefact that would have answered one of the ten questions cheaply, at the time, was either not produced or produced without a shared definition. The review could score the questions. It could not retrofit the evidence.

The specification is rarely the problem. Organisations are good at writing down what they want. They are much worse at writing down how they will know they got it, and worse again at making that contractual.

What changes when part of the work is written by a model

The questions are durable. What is changing underneath them, right now, is what counts as an answer.

Delivery teams on both sides of an outsourcing agreement are starting to write code with assistance. Copilots complete functions. Retrieval-augmented tools answer questions about an unfamiliar codebase from a vector store built over it. A few organisations are running early and carefully scoped autonomous experiments on narrow, reversible tasks. None of that is exotic any more.

Re-read the delivery question. Whether the vendor delivered the output specified in the contract is now a question about authorship and review as much as about output. In an assisted delivery it becomes: can you evidence which parts of this were drafted by a model, on what context, and who reviewed them before they merged. If nobody tracked it, you have not failed a new test. You have failed the old one in a new way, because the acceptance evidence the contract needed does not exist.

The maintainability question moves the same way. Assisted code arrives faster and reads more conventionally than the code a struggling team writes by hand, which makes a design review that judges style rather than structure much less informative than it was. The evidence that carries weight shifts toward what is measured rather than read: coverage, static analysis, dependency structure, and a review record with names on it.

The quality question becomes an evaluation question. Where a model generates or classifies anything that reaches a user, the evaluation set is the acceptance criterion, and like any acceptance criterion it has to be agreed by both parties before work starts, held by the client, and versioned. Lineage does the job the status report used to do, badly. If you cannot say which data a retrieval layer drew on and when that store was last rebuilt, you cannot answer whether the delivered system meets its specification, however good the specification is.

This is the discipline the RealAI Platform is built to make ordinary rather than heroic: evidence produced as a by-product of delivery, not assembled afterwards by a review team working through a document repository. The MLOps investment that looks like overhead at signature is what lets you answer the delivery question in an afternoon instead of a forensic review.

The pre-mortem version

Take the ten questions to the negotiation instead of the post-mortem. For each one, write down the artefact that will answer it, who produces it, on what cadence, and what happens if it is absent. Then check the commercial model rewards producing them. Here the contract carried a penalty regime and no credit regime, so the only lever available pointed at punishment after the fact. It was used, and it changed nothing, because by the time a penalty applies the code is already written.

Ten weeks of grace, two formal escalations and a terminated statement of work later, the subsidiary got a set of clear answers. They were the right questions. They were simply asked at the only moment when nothing could be done with them.

Detail is as recorded in an independent review of a terminated offshore software development engagement at a European banking group's IT services subsidiary: its own ten questions, the summary answer sheet, the contract and statement of work, project artefacts and interviews with both parties. That review produced findings and recommendations, not a recovered programme. Reading its question list forward, as a pre-contract checklist, is ours.