From Batch to Real-Time: Sepsis Detection, Deterioration Alerts, and the Cost of Latency

⤢ Expand

The Problem: Batched Diagnostics Stop at the Admission Door

RealAI's flagship healthcare work — the Cytodeep engagement with the European Health Network — reached 95% diagnostic accuracy across a five-hospital clinical trial, proving that interpretable AI can detect chronic-disease risk months earlier than traditional screening. In that program, time-to-diagnosis fell 4.2 months on average, patient outcomes improved 35% in the intervention group, and healthcare costs dropped 20% through prevention-focused care.

That engagement scored on batched records: a patient's lab history, imaging, and risk factors analyzed periodically rather than continuously. The latency was immaterial; the payoff was prevention. A model that surfaces a high-risk patient in this week's report — instead of last week's — loses nothing, because the clinical window is measured in months.

Sepsis tells a different story. The condition is not a slowly building risk — it is an acute cascade. A patient's vitals cross a threshold and their condition begins deteriorating over a span of hours. In the ICU, the window between the first biomarker signal and irreversible organ failure is short, and it closes fast. A diagnostic system that flags risk on an overnight batch run is too late: the sepsis has already established, the patient has already escalated to higher acuity, and the chance to intervene early — before lactate spikes and organ support becomes necessary — has passed.

This is the latency tax of batch AI in critical care. Architectures optimized for throughput — score a large cohort of historical records overnight — are the wrong shape for real-time demand. Models built to run on a weekly cadence over historical records cannot run continuously against a patient's live vital-signs stream. The compute footprint is wrong, the data pipeline is wrong, and the alert hygiene is untested: clinicians receiving a steady stream of noisy flags will tune the whole system out by the end of a shift, a failure mode known as alert fatigue.

The clinical logic is straightforward. Early intervention — antibiotics, fluids, escalation to the rapid-response team before the patient crashes — is consistently better than late intervention. But that value only accrues if the model runs fast enough to be actionable. A vital-signs alert that arrives after the patient has already deteriorated is not a diagnostic aid; it is a record of what happened.

From Weekly Scoring to Sub-Second Inference: The Architectural Shift

The move from batch to real-time scoring is not a dial-knob. It requires a change in how models are trained, deployed, monitored, and maintained.

1. Model Size and Latency

A Hominis predictive-diagnostics model built for batch processing — the kind that powers the Cytodeep engagement — takes records, labs, imaging metadata, and genetic markers as input, runs through a novel attention-based architecture (the same design that produces interpretable explanations), and outputs a risk score plus the factors driving it. The computation is thorough, and for a periodic batch job that is exactly right.

Real-time vital-signs scoring inverts the priorities. A streaming system must return a usable signal while the clinician can still act on it, which puts a hard ceiling on per-score latency. To get there, models typically have to compress. The common options:

• Lightweight encoder networks: distill the attention-based diagnostic model down to a smaller architecture (quantized, pruned) that runs faster — at the cost of some of the interpretability that made the original model trustworthy.
• Edge-deployed models: run a model on bedside or nursing-station hardware, reducing network round-trips. This requires optimization for embedded environments (constrained chips, limited memory).
• Feature reduction: instead of the full raw vitals panel (blood pressure, heart rate, temperature, oxygen saturation, lactate, white-cell count on a rolling stream), abstract to higher-level physiological state vectors such as early-warning scores. You trade specificity for speed.

The choice in production is often a hybrid: a lightweight fast-path scores routine vitals continuously and flags potential deterioration; when a threshold is crossed, the full diagnostic model runs to explain why and to route the alert. This two-tier approach keeps the alert itself fast while preserving the clinician-facing explanation that builds trust — the same interpretability principle that carried Cytodeep through regulatory approval and physician adoption.

2. Data Streaming and Continuous Scoring

A batch system ingests a file, scores a cohort, and writes results once per cycle. A real-time system must continuously pull vitals from a bedside monitor (or a hospital data lake) and score them as they arrive.

The plumbing is non-trivial:

Process flow · hover a step to trace itExpand

A typical architecture uses a message broker to buffer vitals from multiple beds, a stream-processing layer to compute rolling windows (has blood pressure been climbing over the last several readings?), and a model-serving framework to run inference across a hospital floor. Each of these layers adds latency and operational complexity, and engineering discipline is what keeps the end-to-end path fast enough to matter.

3. Alert Prioritization and Fatigue Prevention

A batch system produces one output per patient per run: a risk score. A real-time system produces many signals per patient per day — a fresh vital-signs assessment on every cycle, plus threshold crossings, plus the results of the full diagnostic model.

Left unfiltered, that volume drives alert fatigue. When clinicians are paged for low-value signals, they begin dismissing alerts unread, and the system becomes worse than useless — it becomes noise that masks the real event.

The antidote is alert prioritization:

• Rank alerts by actionability: a high-confidence signal with a recommended action routes straight to the rapid-response team; a low-grade, likely-benign signal is written to the chart for context without triggering a page. This mirrors the governed alerting pattern RealAI uses in production — a high-risk signal routed to the right team, with every alert carrying its driving factors.
• Combine signals to suppress noise: a single elevated heart rate is not sepsis; elevated heart rate plus rising lactate plus falling blood pressure is far more specific. Composite scores that require multiple concordant signals before escalating cut the false-alarm rate.
• Learn alert utility from outcomes: every time a clinician acts on an alert and an outcome is recorded — antibiotics given and sepsis confirmed, or the patient assessed and deterioration ruled out — that feedback tunes the threshold for the next cycle. A model that ignores feedback and keeps firing the same noisy alerts is actively training clinicians to distrust it.

The systems that sustain clinician adoption tend to implement all three together: explainable scoring, composite thresholds, and continuous retraining on action logs. Adoption is earned through transparency, not raw accuracy — the same lesson Cytodeep learned in the clinic.

Real-World Anchor: Extending Cytodeep Toward Real-Time

The Cytodeep case study — 95% diagnostic accuracy, 4.2 months earlier detection, 20% cost reduction, with 89% sensitivity and 92% specificity across the five-hospital trial — was built on batch scoring. A patient's record arrived at the hospital's data lake, the model scored it against chronic-disease risk, and a periodic report surfaced high-risk patients for preventive screening. The federated-learning design meant patient data never left its source hospital, and the attention-based architecture surfaced the specific risk factors behind every assessment.

Extending that foundation toward real-time ICU alerting would require a fundamentally different deployment:

1. Retraining cadence shifts from periodic to continuous. As sepsis cases are confirmed or ruled out, those outcomes flow back into retraining. A model trained on last year's cohort will not recognize a new pathogen or a shift in patient acuity; continuous retraining keeps it honest.

2. Infrastructure moves from batch compute to streaming infrastructure. Instead of a periodic job over historical records, the system runs continuously over live bedside data. The cost profile changes: batch processing is cheap per patient because it amortizes over a large cohort; streaming demands serving capacity held ready to score one patient's vitals at low latency, continuously. Hospitals must budget for that operational overhead.

3. Regulatory oversight intensifies. A batch diagnostic scored on stable historical data is easier to validate than a real-time system scoring on noisy vitals in a high-acuity setting, where alerts route into time-critical clinical decisions. Regulators expect continuous monitoring, drift detection, and documented retraining procedures for real-time clinical systems. The governance overhead is substantial — and, as Cytodeep showed, governance done well is what wins approval rather than what blocks it.

4. Alert hygiene must be proved before go-live. Before a hospital deploys a real-time alerting system at scale, it runs a validation phase: shadow alerting, where alerts are generated but not shown to clinicians, to measure accuracy and false-alarm rate against ground truth. Only after that cohort is analyzed is the system handed to the rapid-response team.

Why Latency Is the Binding Constraint

In batch diagnostics, accuracy is the headline number — and Cytodeep's 95% is the proof that interpretable models can clear the clinical bar. In real-time critical care, accuracy is necessary but not sufficient. A perfectly accurate model that returns its verdict after the patient has crashed delivers no clinical value at all.

That reframes the engineering problem. The question is not only "how accurate is the model?" but "how accurate is the model within the window where action still changes the outcome?" Every layer that adds latency — the broker, the stream processor, the model server, the network hop to the cloud and back — narrows that window. The hybrid fast-path/full-model pattern exists precisely to protect it: get a fast, actionable signal out first, then attach the slower, richer explanation.

The economics follow the clinic, not the other way around. Early sepsis intervention avoids the most expensive and most dangerous outcomes — prolonged ICU stays, mechanical ventilation, vasopressor and dialysis support — while late intervention incurs all of them. A real-time alerting layer that buys clinicians earlier, well-targeted warnings is therefore both the clinically and the financially superior path, provided the alerts are trusted enough to be acted on. Each hospital's specific savings depend on its case mix, baseline detection rate, and current cost structure, and should be modeled against local data during the assessment phase rather than assumed from industry averages.

95%

Diagnostic accuracy (Cytodeep)

89%

Sensitivity in trial

92%

Specificity in trial

4.2 mo

Earlier detection

Where to Start: The Assessment

The move from batch to real-time turns on three early decisions:

1. Which acute condition to target first? Sepsis is high-impact but data-intensive — it needs accurate vital-signs histories, lab results, and infection-status records. Generic patient deterioration (early-warning scores over standard vital-sign thresholds) is easier to model but lower in specificity. Start with the use case where your hospital has the cleanest data and the clearest outcome ground truth: did the patient develop sepsis, yes or no?

2. Where to deploy the model? Bedside monitor (edge)? Central nursing station? A cloud system pulling from the EMR in real time? Edge deployment is faster but depends on vendor partnerships and hardware refresh cycles; cloud is more flexible but adds network latency. Map your current infrastructure and latency constraints before committing.

3. How to measure alert adoption? Before any hospital-wide launch, define the success metrics: false-alarm rate (what share of alerts were clinically actionable?), response time (how long from alert to clinical assessment?), and outcome capture (were confirmed and ruled-out cases both logged?). These metrics drive the retraining loop and prove value to the hospital's leadership.

RealAI's Assess phase typically runs 4–6 weeks. The team audits vital-signs data quality, defines the target condition and outcome cohort, runs a retrospective validation (does the model score historical data correctly where ground truth is known?), and delivers a ranked roadmap of use cases and deployment architectures. From there, Transform builds the streaming pipeline and serving layer; Sustain runs the continuous monitoring and retraining that real-time clinical AI demands.