CS1-4B, a small model for reading patient records: 13 of 14 tasks won against models up to 27 billion parameters, but labels from other LLMs and no prospective validation
A team from Mass General Brigham and MIT presents Clinical System One (CS1-4B), a small 4.2-billion-parameter language model that answers typed questions about patient records in a single forward pass, with a probability attached to each answer. On 963,408 held-out patients from the source health system it beats comparable open models of up to 27 billion parameters on 13 of 14 tasks, and it approaches their scores on 21 public datasets at a much lower compute cost. The engineering result is solid, but much of the training and evaluation labelling comes from other language models, and nothing has been validated in clinical use.
The context
A hospital patient record holds years of notes, results and prescriptions. To answer "has this patient ever received an anticoagulant?" or "is this patient eligible for this clinical trial?", the usual approach is to hand the record to a large language model (LLM: a network trained to predict text, which can also follow instructions). It is accurate but slow and expensive, because each question rereads the whole record.
The authors, including Faisal Mahmood (Mass General Brigham AI Institute, Harvard Medical School, MIT), argue for a division of labour inspired by the psychological "system 1 / system 2" distinction: a small, fast model reads records and returns short typed judgments, while a large generative model asks the questions and assembles the evidence. This is an arXiv preprint (2610.27099, submitted 19 October 2026, cs.AI), not peer reviewed.
The method
Architecture. CS1-4B has 4.2 billion parameters and is initialised from Decision 2.0 Nox-4B, itself derived from Qwen3.5 (an open language model) with 24 Gated DeltaNet linear-attention layers and 8 full-attention layers. "Attention" is the mechanism that lets a model relate the words of a text to one another. The model is fully fine-tuned (all weights are updated). Each document is encoded once; each question is a short segment that attends to the document but not to the other questions, so about forty can be asked at once. An output head gives one score per option, converted to a probability. Questions come in three types: choice (2 to 11 options), yes/no, and ordered score. They are written at request time, so a new criterion needs no retraining.
Data. The Klaros-12M corpus holds 12,416,448 document-question pairs from 3,643,798 Mass General Brigham patients (24 hospital sites, median follow-up 12 years, 7.47 million documents). Labels come from three sources: "soft" labels (probabilities) from a teacher model, GLM-5.3-Flash (320 billion parameters in total, 18 billion active), for 4.51 million note pairs and 2.46 million report pairs; exact labels computed by rules from structured data (5.01 million timeline pairs); and outcomes actually observed after the decision point (433,000 forecasting pairs). Training took 26.4 hours on eight NVIDIA B300 GPUs, or 211 GPU-hours. No public or outside-institution data was used for training.
Evaluation. Internally, the MGBQA-1M benchmark covers 963,408 held-out Mass General Brigham patients, with balanced accuracy as the main metric (the mean of sensitivity and specificity, useful when classes are imbalanced). Externally, 21 public datasets are used for evaluation only: MIMIC-IV, eICU-CRD, EHRSHOT, MedAgentBench, BRIE, TCGA, HISTAI, MIMIC-CXR-VQA, LongHealth and others. Comparators are ten open models from 0.6 to 27 billion parameters, including the Decision 2.0 family from Kai-0.6B to Vega-27B.
The results
ML metrics. Averaged over the six internal task types, CS1-4B scores 89.7, against 73.3 for its starting model Nox-4B and 81.7 for the best open model (Vega-27B). It leads on 13 of 14 independent tasks: timelines 96.6% vs 86.8% for JEV-27B, clinical definitions 92.1% vs 85.0%, outcome forecasting 75.6% vs 69.4%, medication status 95.0% vs 82.6%. The one exception is cancer staging from reports (86.9% for pplx-decider vs 86.6%).
Externally the mean is 86.7 vs 88.6 for Vega-27B: CS1-4B approaches the largest model without matching it. It beats it on five datasets (for example EHRSHOT-Prognosis, mean AUROC 0.766 vs 0.681 for Clef) but trails by 7.4 to 12.8 points on three clinical-definition sets and by 1.1 to 5.9 points on nine of the eleven note and report sets. Inference is 6.4 times faster than Vega-27B (1.80 s vs 11.54 s for 40 questions on a 2,900-token note).
Calibration. Calibration measures whether a stated probability matches the real frequency (the expected calibration error, ECE, is zero when ideal). It ranges from 0.012 to 0.052 across task types. But for forecasting 27 outcomes, the mean predicted risk is 0.39 against an observed 0.19: among 1,000 patients, the model would announce about 390 events where about 190 occur. A simple per-outcome shift brings the error from 0.203 down to 0.014.
Clinical translation: trial screening. Paired with a general LLM that never reads the full record, CS1 screened 17 ClinicalTrials.gov trials across 1,000 MIMIC-IV patients. On structured criteria, accuracy is 0.884 vs 0.869 for an LLM reading everything. On criteria found only in notes, it is 0.766 vs 0.796, a difference of −0.030 whose confidence interval excludes zero. For ranking patients, nDCG@20 (quality of the top-20 ranking) is 0.64 vs 0.11 for keyword search, with 3.5 judged-eligible patients in the top 20 vs 0.8. The cost: 610 LLM tokens per patient vs 421,136, or 690 times fewer, excluding CS1's own compute.
What is good
Scale and a clean evaluation split. Training uses 3.6 million patients, internal evaluation 963,408 distinct patients, and the 21 public datasets are used only for testing. The authors also report their losses: cancer staging, notes and reports, external clinical definitions.
Calibration measured and criticised. The authors publish ECE by task type, then acknowledge the overestimation of risk in forecasting and quantify the effect of recalibration. Few clinical language model papers do this.
An honest cost-accuracy trade-off. The −0.030 gap on note-derived criteria is given with its confidence interval, next to the 690-fold token saving. Readers can weigh it themselves.
What is less good
Labels produced by other models. Part of the training labels come from a teacher model, and several evaluation keys are written by models (HISTAI by the teacher, BRIE by an open LLM, ReXVQA by an LLM pipeline, plus 14 secondary tasks). The model may thus learn to reproduce its teacher's errors without the test revealing them. The reasoning annotator agrees with human keys on 94-98% of items on three benchmarks, but only 90.6% on MIMIC-III phenotypes, below the majority-answer baseline (92.5%). Failure modes: circular leakage, misleading metric.
One health system, no prospective validation. All training data comes from Mass General Brigham, whose coding and documentation habits may not transport. The public datasets are mostly emergency and intensive care (MIMIC-IV), and outcomes may be missing when care happened elsewhere. The authors themselves state the model is not prospectively validated and must not decide alone. Failure modes: population bias, distribution shift.
LLM-judged screening, a weak comparator, limited reproducibility. The three worked whole-record answers are judged blind by GLM-5.3. Agreement between two passes of the judge is 0.84 (weighted kappa), but that measures consistency, not agreement with clinicians. The 0.11 nDCG comes from a keyword baseline (BM25), a weak comparator. Weights will be released only after institutional approval, no code is announced, and funding and conflicts of interest do not appear in the text consulted. Failure modes: biased comparator, limited reproducibility.
What it changes
For the research community, the paper supports an architectural idea: small specialised models for repeated reading, large models for planning. It also offers an evaluation method worth copying: per-task calibration, quantified cost savings, documented losses. The open question is what remains when labels come from humans.
For clinicians, nothing changes today: the tool is neither prospectively validated nor available, and it targets data, clinical research and coding teams first. In the medium term, trial eligibility screening is the most concrete path, with human review essential.
For patients and the public, the stakes are indirect: clinical trials possibly recruited faster if screening delivers. It is also worth remembering that probabilities displayed by a model are usable only if calibrated, and CS1's were not for forecasting without correction.
Going further
Paper: A general-purpose System One decision model for clinical records, arXiv 2610.27099, DOI 10.48550/arXiv.2610.27099. Public datasets used for evaluation: MIMIC-IV, eICU-CRD, EHRSHOT, MedAgentBench, LongHealth. Weights: release planned under credentialed access.
Editorial transparency note: this reading was produced with the help of AI tools from the preprint text; translations are automatic.