Error Swallowing

Every reliability review starts by hunting the crash, because a crash is generous. It arrives with a timestamp, a line number and somebody's attention already attached. The defects that cost the most sit at the far end of that scale, in code that catches a failure, declines to raise it, and continues as though nothing happened.

An independent assessment I led on a front-office advice application at a European retail and private banking group put a name on the pattern. The review worked through a checklist of quality properties, binding its findings to references in the client's defect tracker. Against exception handling it logged four findings across three failure modes: one for error swallowing, two for handling applied in some paths and absent in others, and one for an exception that reached nothing at all.

No narrative was written against that page. The categories and the tracker references are the whole record, and that is a drafting state rather than a verdict: thirteen of the document's eighteen pages still carry the template's unfilled word in the findings box. What survives is the taxonomy and the count. The reasoning that produced them sits nowhere a later reader can reach.

Rank those three failure modes by how loud they are, and the ranking runs exactly opposite to the harm.

The quietest defect is the expensive one

An unhandled exception is the loudest of the three. Something visibly breaks, a user notices, a log line exists somewhere, a ticket gets raised, an engineer reads a stack trace. It is unpleasant and cheap, because the system told on itself.

Handling applied in some places and not others is worse, because it manufactures confidence. A reviewer who finds a guard three lines above the code in question assumes the neighbourhood is covered. What makes error handling readable is consistency rather than presence, and inconsistent handling reads as covered until the path nobody guarded is the path that runs.

Error swallowing is worst, and it is worst by a distance. The failure is caught deliberately and then discarded. Control flow resumes. The function returns a default. The response serialises. The screen renders. Nothing raises, so nothing logs, so nothing alerts, so no ticket is opened, so no metric moves. The availability graph stays flat and green the whole time, because in every sense that graph is capable of measuring, the application was available.

Put that in an advisory workflow, which is where this application lived. An advisor is sitting with a client. A step that refreshes or enriches a position fails and is swallowed. The field renders blank, or renders the value it held before, or renders a default. Nothing on the screen distinguishes a number that is current from a number that is merely the last one anybody had. The conversation continues, advice is given, and the file looks identical to every file around it.

The question a supervisor asks about that months afterwards is rarely whether the system failed. It is when anybody knew.

Three places the signal could have surfaced

The same document shows how little in the estate was positioned to answer it.

At runtime, logging was assessed as its own property and the record carries nothing against it: no findings, no tracker items, no remark. The document does not say whether that means the control was clean or was never reached, and both readings land in the same place. The evidence that would have contradicted a swallowed error is not available either way.

At build time the picture is measurable. Server-side coverage was measured at about 14 percent with a named tool. Front-end coverage could only be estimated, probably below 10 percent. The tests that existed were described as mostly null and length checks, which the reviewer gave as the reason most of them were not very useful. One qualification comes from the formatting rather than the words. That entire passage is set in red, and the only other red in the draft sits on items the author still meant to check. No colour key is declared anywhere, so reading it that way is ours, but it puts the fourteen percent closer to a measured order of magnitude than to a signed-off result.

A null check is precisely the assertion a swallowed error passes. The catch block returns an empty object, an empty list or a default, and the test confirms that something came back and has a length. Green, every run, on the exact defect the suite exists to prevent.

So the runtime signal was suppressed by construction, the logging evidence is absent from the record, and the assertions could not separate a real value from a substituted one. Three observations from three parts of one review, describing a single blind spot: this codebase could not detect its own failure at runtime and could not detect it at build time either.

Remediation follows detectability, not severity

The only repair recorded anywhere in the document sits on the concurrency finding: front-end race conditions and shared server-side state, described in the text as easily identifiable, fixed roughly three months after the assessed build.

Nothing comparable is recorded against the structural findings, the coverage measurement or the exception handling. I would not claim those went unfixed; the document does not say. What it shows is which class of defect generated enough of a trail to be worth a fix note, and it was the class that announced itself.

A swallowed error announces nothing, which puts it at the bottom of every queue it enters. In practice it enters none.

The cadence compounds it. The assessment is stamped throughout as a point-in-time reading of one build, written up roughly seven months after that build. For a defect class whose defining property is that it emits no signal between reviews, periodic and never are close to the same setting.

The pattern did not stay in the past

I would leave this as engineering history if agent harnesses had not made it considerably easier to commit.

A harness wraps each tool call in a catch so that one failing call does not end the run. That is sensible engineering, and it is also error swallowing with a longer reach, because whatever the catch returns goes back into the loop as text and a model reads text. A retrieval step that returns nothing looks, on the next turn, like a topic with no relevant documents. A permissions failure looks like an empty set. The agent does not stall on missing evidence. It reasons over what it received and produces something fluent, and the failure has been converted into an answer.

Graded autonomy does not rescue this, because escalation fires on a condition and the condition was caught and dropped three layers down. Evaluation sets do not rescue it either: they score outputs against expectations, and an answer built on a silently empty retrieval is frequently plausible. It is wrong in a way the run itself has no means of noticing.

That is why we build step-level status into the Hominis loop rather than only into its logging, and why the RealAI Platform treats every tool result as a pair, an outcome and a state, so a later turn can tell no results apart from the search did not run. The distinction is cheap to carry and unrecoverable once dropped.

There is a supervisory edge too. Model risk management has asked the same thing in older language for as long as banks have had models: name the control, show that it detects, name who sees the detection. Every record-keeping and oversight expectation built on top of that assumes a system can account for what it did and where it failed. A workflow that catches and continues fails all three while passing every uptime check in the building.

A swallowed error does not degrade the system. It degrades the output while leaving the system's own account of itself intact, which is why the availability graph stays green for the entire time the answer is wrong.

What we would change, and none of it needs a model

Five rules, in order.

No catch without a destination. A caught failure either changes control flow or produces a record with an owner. Discarding it is a decision, and decisions get written down where a reviewer looks.

Status travels with the payload. Every step returns its outcome and its state, and a partial result is labelled partial rather than handed on as a smaller complete one.

The surface tells the truth. Where a value is missing because a step failed, the screen says so in the place the value would have been. A blank field and a failed field must never look alike to somebody about to advise a client on the strength of them.

Test the failure path, and assert on content rather than existence. If the only way to pass is to return the right value, a default cannot pass, and the class stops being invisible at build time.

Every assessed control reports a state: checked and clean, checked and defective, out of scope with a reason, or not yet run. A blank is the one output nobody can audit, and this record carries several.

4

Exception-handling findings logged, across three failure modes

1

Of those recorded as error swallowing

0

Logging findings recorded, clean or unexamined not stated

~14%

Measured server-side coverage, in a passage the draft still marks unchecked

None of it is expensive, and all of it has to be true before an agent acts on the output of a step it cannot verify ran.

Findings are as recorded in an independent assessment of a front-office advice application at a European retail and private banking group: a quality checklist bound to the client's own defect tracker, plus one coverage measurement. It is a working draft rather than a signed-off deliverable, and it produced findings against a single build, not delivered results. Reading the exception-handling categories as a signal-visibility problem is ours.