Skip to content
Hominis Agentic OS · early access program now openJoin the waitlist
RealAI

Case studiesBanking technology

Case study
Banking technologyA European banking group's IT services subsidiary

One cache used as shared memory, written asynchronously with no completion signal, sat underneath the erratic behaviour nobody could reproduce

A European banking group's IT services subsidiary commissioned a review of an offshore software build it had terminated, covering the sourcing arrangement, the way the project was run, and the code itself. The technical findings included the usual inventory of duplicated code, thin typing and unhandled exceptions. One finding sat underneath most of the rest. The application's client-side cache was not being used for efficiency; it was being used as shared memory between components, a global blackboard, and server data was written into it asynchronously with no signal to the reader that the write had completed. Reads and writes therefore resolved in whatever order timing produced, sometimes returning the old value and sometimes the new one. The review's own concurrency section pointed straight back at the caching section rather than listing anything separate. What this engagement produced is a set of findings and recommendations for both parties, not a repaired system.

10 to 15%Approximate test coverage the review found across the delivered stacks
Client
A European banking group's IT services subsidiary
Duration
Post-termination review of an offshore build, findings and recommendations
AI · RIDGE E59.1 N62.5ρmax 1.00
100KBLargest data packages the review observed crossing the wire on one exchange
~10 weeksBetween the two builds reviewed, with no improvement enough to allow acceptance testing
140 PDEstimated in the statement of work for a design document with no evidence it was written

A cache is supposed to be an optimisation you could delete. Take it out and the application still produces the right answer, only slower. The moment removing it changes what the application does rather than how quickly it does it, the thing in the code is no longer a cache. It is a shared data store that two parts of the system are using to talk to each other, and nobody has written that down anywhere a reviewer would find it.

A European banking group's IT services subsidiary commissioned a review of an offshore software build it had ended early. The scope covered the sourcing arrangement, the way the project had been run, and the software itself, and included a technical read of the codebase plus a difference test between two builds taken roughly ten weeks apart. The subsidiary had been reporting behaviour nobody could reproduce on demand: values that were not there when the screen expected them, results that differed between runs on identical input, a build the client would not put in front of users. The review found the ordinary inventory of software problems, and one decision sitting underneath most of the rest.

The challenge

The application cached data on the client side. Part of the caching code had been rewritten during a refactoring exercise and part had been left on the older design because of time pressure. On its own that is a maintenance note. Three properties turned it into the finding of the review.

The first is that caching was not applied transparently. It was applied case by case, which means no reader of the code could say from the outside which data came from the cache and which came from the server. Anyone reasoning about a screen had to reconstruct it per call site.

The second is the one that matters. The local cache was not being used for efficiency. It was being used as shared memory between components: one part of the application wrote a value into it, another part read that value out, and that was the channel over which the two coordinated. The report named it a global blackboard, which is the correct name for it and also the reason it never appeared in any design artefact. Nobody designs a blackboard. It accretes one convenient write at a time, and each of those writes is individually reasonable.

The third is that data obtained from the server was written into that store asynchronously, and the code that consumed the data had no way of knowing whether the write had arrived. There was no completion signal, no version, no state flag. A reader took whatever sat in the slot at the instant it looked.

Put the three together and the behaviour stops being mysterious. Data is not always available at the moment it is expected. Reads and writes resolve in whatever order timing hands them, so the same code path returns the old value on one run and the new one on the next. That is a race condition, and it is not an incidental defect in a component. It is the direct consequence of using an undeclared store as a coordination channel.

The strongest evidence that this was the root and not a symptom is how the review's own performance section handled concurrency. It did not list a separate set of concurrency defects. It pointed back at the caching section. Everything the reviewers could say about the intra-component coordination in the application had already been said while describing the cache.

The same shape appears elsewhere in the findings once you know to look for it. Session state was being held in static variables on the server, so with multiple sessions active one user could reach another user's data, which in an application handling client information is a security problem before it is a correctness problem. Modules held circular references, so the order of loading and executing, which varied with factors outside the code, decided whether a piece of code could run at all. Errors were swallowed by guard conditions that tested whether the next statement was executable and skipped it silently when it was not, which removed the visible exception and left the missing effect. In each case something that determines behaviour passes between parts of the system through a route that is not declared, not owned and not checked.

The approach

The review worked from the contract and the delivery artefacts, from interviews with both sides, and from the code as it stood on the shared environment, with a second read of a later build to test whether quality had moved. It had not moved enough to support acceptance testing. Automated coverage across the two main technology stacks was somewhere between ten and fifteen percent, so almost none of this had a chance of being caught by a test. Data model handling in the front end had no consistent concept behind it, and single exchanges were moving packages as large as a hundred kilobytes across the wire, with a data context passed back and forth on each call. The statement of work had estimated a hundred and forty person-days for a design document setting out coding standards, design principles and the rationale for the development approach, and the reviewers found no evidence that document existed.

That missing document is the same finding as the cache, one level up. The recommendations put to both parties were the sensible ones: agree design and coding standards upfront, give explicit guidance on design patterns, put automated tooling in the pipeline to check compliance, and run peer review seriously. All four are worth doing and none would have caught this. A blackboard write passes peer review, because the file it appears in is fine. A static analyser accepts it, because the call is legal. The defect is not in any file. It is in the relationship between two files that no artefact describes.

What would have caught it is a declared contract for anything shared: what lives in it, who writes each entry, who reads it, and what a reader is guaranteed at the moment it reads. Three lines per entry. This is exactly the discipline our Platform work starts with, because a coordination channel that nobody has to declare is a channel nobody can review.

The outcome

This engagement produced findings and recommendations delivered to the client and the supplier, not a repaired application. The build was already over. The value was a diagnosis specific enough that neither party could reduce it to a difference of opinion about quality, and a recommendation set naming where the process had to change rather than who to blame.

The reason to publish it now is that the pattern has moved rather than disappeared. In a machine learning pipeline the shared store has a product category and a budget line. A feature store written by one job and read by an inference service is the same arrangement: populated asynchronously, consumed by a reader with no way to establish whether the value it just read reflects the write it is assuming. When the deployed model then returns a different decision on the same input twice, the investigation starts on the model, and the model is fine.

An undeclared shared store used as a coordination channel between components is the same anti-pattern whatever decade it appears in. The only thing that changes is what we call the store.

NEXT STEP

Ready to make AI real?