Jev, a decision model at $0.02 per 1,000 answers: offloading structured clinical tasks from LLMs, as long as you trust it only when it is sure

A team from the Clalit Research Institute and Ben-Gurion University compared Jev, a non-generative "decision model", with ten large language models (LLMs) on closed-answer clinical tasks. Alone, Jev is clearly less accurate: 45.4% accuracy for colon cancer stage, against 94.2 to 100% for the LLMs. But its "fully confident" answers are almost always right, and a hybrid set-up in which it answers first and hands over to the LLM when unsure stays within 1.7 points of the LLM alone while cutting cost by 33 to 44% with flagship LLMs. The idea is sound, but the preprint relies on synthetic pathology reports and on a model whose architecture is not disclosed.

The context

Many clinical uses of LLMs are in fact closed tasks: say whether a patient is at risk, pick one option among a few, place a case on an ordered scale (a tumour stage, for example). A general-purpose LLM, which generates text word by word, is powerful but costly and slow, and its outputs are not calibrated probabilities. The authors therefore ask an engineering question: can part of these tasks be given to a much smaller model that directly returns a probability, a category or a level, with a confidence indicator, calling the LLM only for the rest?

The preprint was posted on medRxiv on 12 October 2026. It has not been peer reviewed and states that it should not guide clinical practice.

The method

The model tested. Jev (version 1.13) is developed by the company TypeSafe AI. It is a decision model: it is asked a typed question about a case and returns a probability (yes/no), a probability per option (choice) or a probability per level of an ordered scale (score). Architecture, parameter count and training data are not described in the paper.

Three families of tasks. (1) Risk of death: ten-year mortality in 12,697 adults aged 40–79 from the NHANES cohort (1999–2008, linked to death records), and ICU mortality in 1,297 stays from the eICU demo database. (2) Category choice: 180 patient questions written by the authors on 14 Choosing Wisely recommendations (125 low-value, 55 indicated), plus 1,571 exam questions (MedQA 1,273, Medbullets 308). (3) Ordered scale: 240 synthetic pathology reports of colon cancer, scored with the AJCC 8th edition, and 561 eICU records for acute kidney injury stage.

The comparators. Ten LLMs (five flagship, five smaller, including MedGemma 27B), called with default settings through the OpenRouter platform. For mortality, a logistic regression is also used as a reference.

The statistics. Confidence intervals by bootstrap (2,000 paired resamples). For the hybrid, a non-inferiority test with a 5-percentage-point margin: it checks that the hybrid does not lose more than 5 points against the LLM alone. Holm's correction handles multiple comparisons.

The results

ML metrics. For ten-year mortality, the AUROC (the probability that the model gives a higher risk to a person who died than to a randomly drawn survivor) is 0.822 (95% CI 0.813–0.832) for Jev, 0.837 to 0.846 for the flagship LLMs and 0.854 for logistic regression. Calibration, the agreement between announced and observed risk, is poor: mean predicted risk 26.3% against 15.8% observed; LLMs range from 18.5 to 25.2%. In intensive care, AUROC is 0.836 against 0.845 to 0.880.

On patient questions, balanced accuracy is 87.7% (CI 82.7–92.1) against 98.4 to 100%; on exam questions, 85.6% against 91.2 to 95.9%. For colon cancer stage, 45.4% against 94.2 to 100%: 9 of the 63 stage III reports are placed in stage 0–II. For acute kidney injury, 54.4% against 90.9 to 99.5%.

Developer-recommended inputs. With medical context (definitions of each option) and task decomposition (the stage is computed by code from four sub-questions), Jev rises to 75.8%, 82.1%, then 85.8% for colon staging; with KDIGO criteria it reaches 91.8% for kidney injury. LLMs given the same aids reach 98.8 to 100% and 97.9 to 100%. Mortality calibration does not improve.

The confidence signal. Across all tasks, 1,351 of Jev's 1,354 fully confident answers (99.8%) are right; none of its 34 staging errors was fully confident. With the recommended inputs, it is fully confident on 48% of pathology reports.

The hybrid. Jev answers 116 of the 240 reports alone (48.3%); the rest go to the LLM. Hybrid staging accuracy is 95.8 to 100%, within 0.0–1.7 points of the LLM alone, and all ten pairings satisfy the 5-point non-inferiority margin. Cost falls to 56–67% of the LLM alone (33 to 44% less) with flagship LLMs, but two pairings with small models cost more. The least accurate LLMs gain 21 to 42 points thanks to Jev. In addition, Jev answers in 0.90 to 1.02 seconds (median) against 2.5 to 36.8 s, for $0.02 to 0.04 per 1,000 answers against $1.56 to 15.08.

Clinical translation. Out of 1,000 stage III reports, "standard" Jev would place about 143 in stage 0–II (9 of 63), an error that can change the adjuvant chemotherapy decision: this is exactly why it should not be used alone. For mortality, a mean announced risk of 26.3% instead of 15.8% amounts to predicting about 263 deaths per 1,000 adults when about 158 are observed (editorial calculations). For the hybrid, out of 1,000 reports about 483 would be settled by Jev without calling the LLM, with the accuracy figures above.

What is good

A well-posed, quantified engineering question. The paper does not claim Jev replaces LLMs: it first reports that it is less accurate and poorly calibrated, then shows under what condition delegation is possible. Cost, latency and accuracy are measured together across ten pairs.

A confidence signal evaluated in its own right. The paper reports how many fully confident answers were wrong (3 of 1,354) and what share of correct answers were fully confident (56% for staging), rather than settling for overall accuracy. This is what makes the hybrid plausible.

A careful, partly reproducible comparison method. Pre-set non-inferiority margin, paired bootstrap, multiple-comparison correction, logistic regression as a yardstick. Code, prompt templates and aggregate results are published on GitHub (MIT licence) and archived on Zenodo; NHANES, eICU, MedQA and Medbullets are public or controlled-access sources.

What is less good

Synthetic, author-written data (population bias, favourable comparator). The 240 pathology reports are synthetic and only a sample was adjudicated by a physician; patient questions were written by the authors and validated by a physician author. Real reports are often more heterogeneous and less well formed: the 48.3% coverage and the hybrid's accuracy could degrade on them. ICU data come from the demo database, and the kidney injury sample is enriched in cases.

An opaque model and a developer link to clarify (conflict of interest). Jev belongs to TypeSafe AI; architecture, size and training data are not described, and weights are not stated to be available. The authors declare no conflict of interest and no external funding, but in the passages consulted, any relationship between the authors and the company is not specified. Reproducing the patient and pathology tasks requires a request to the authors.

Reliable confidence… on these datasets (misleading metric, possible leakage). Exam questions are public and may have appeared in the training of LLMs, or of Jev. The 99.8% of correct confident answers is an aggregate figure: three confident errors exist, and we do not know how their frequency changes out of distribution. Finally, mortality is excluded from the hybrid (a probability cannot be "right" for an individual), and costs rely on September 2026 list prices and time estimates.

What it changes

For the research community. A reusable protocol: measure not only a small model's accuracy but the reliability of its confidence indicator and the accuracy of a cascade with a non-inferiority margin. The logical next step is to test on real hospital records and against competing decision models, with training described in detail.

For clinicians. Nothing changes today: this is a preprint that rules out clinical use. In the medium term the lesson is architectural: for classification or staging tasks in a hospital information system, a light model followed by an LLM can reduce cost and latency, provided the confidence indicator is audited on the institution's own data.

For patients and the public. No direct effect. Keep one reflex: an AI tool that "knows when it does not know" is more useful than a slightly more accurate tool that always answers with the same assurance.

Going further

Preprint: Ariel D., Laban G., Endelman S., Balicer R., Dagan N., Cohen S., "Decision models can offload structured clinical tasks from large language models", medRxiv, 12 October 2026 (DOI 10.64898/2026.10.09.26365130). Code and aggregate results: GitHub repository "deanariel-md/decision-model-offload" (MIT licence) and Zenodo archive (DOI 10.5281/zenodo.23070012). Reference data: NHANES and its linked mortality files, eICU (PhysioNet), MedQA, Medbullets.

Editorial transparency: this article was produced with the help of artificial intelligence tools from the preprint's text; figures come from the paper, except the per-1,000 conversions, calculated by the editorial team. The English, Spanish and Chinese versions are machine translations.