Neuro-VL: a vision-language model trained on nearly one million Michigan brain MRI and CT exams outperforms GPT-5.4 on a benchmark built by its authors, with no comparison against radiologists

A University of Michigan team presents Neuro-VL, a vision-language model that reads complete brain MRI and CT examinations, answers questions, compares an exam with the previous one and drafts a report, after training on 961,683 exams from routine care in a single health system. On a benchmark of nearly 32,000 questions covering an internal cohort and 24 public datasets from 21 countries, it scores on average 15.9 points higher than GPT-5.4 and is far more resistant to questions that suggest an abnormality that is not there. The work is large and its weights are released, but most of the evaluation relies on questions and a judge generated by other LLMs, with no comparison against neuroradiologists: this is a solid laboratory result, not yet evidence of clinical usefulness.

The context

Neuroradiology is one of the most studied fields in medical AI, yet most deployed tools remain single-task: one algorithm flags a hemorrhage on CT, another segments a tumor on MRI. Vision-language models (VLMs, which pair an image encoder with a large language model able to produce text) promise something different: a single system that looks at the whole exam and talks about it, like a radiologist dictating a report.

Two obstacles stand in the way. General-purpose models such as GPT-5.4 or Gemma 4 were designed for 2D images and see an exam slice by slice, with no native representation of the volume. And specialized models are often trained on public corpora of a few thousand exams. The same laboratory, the Machine Learning in Neurosurgery Lab led by Todd Hollon, had already published NeuroVFM, a foundation model (a model pretrained on a very large amount of data and reusable for many tasks) learned from Michigan Medicine health-system data. Neuro-VL is its conversational extension. The preprint, posted on arXiv on 11 November 2026 (41 pages), has not yet been peer reviewed.

The method

Training data come from routine care at Michigan Medicine: 961,683 exams (421,457 MRI and 540,226 CT) from 440,236 patients, acquired between 1995 and 2025 on scanners from several manufacturers. The train/test split is temporal (cutoff in July 2025) and made at the patient level, so that no patient appears on both sides, which prevents a classic form of data leakage. In total: 911,506 training exams, 1,552 validation exams and 48,625 internal test exams.

The architecture combines a ViT-Large visual encoder (a 24-layer vision transformer, which cuts the image into small blocks and learns the relationships between them) with the Qwen3-8B language model (8 billion parameters). Each volume is cut into 6 × 16 × 16 voxel cubes, each turned into a numerical vector, a visual "token"; a small intermediate network groups these tokens four by four before passing them to the language model. The encoder is first pretrained using summarized radiology reports, then the whole system is fine-tuned on about 740,000 question-answer examples generated automatically by GPT-5-family models from radiologists' reports. A longitudinal variant, Neuro-VL-Longitudinal, additionally learns to compare an exam with the previous one using about 155,000 exam pairs (230,000 examples, also generated by GPT-5).

Evaluation relies on NeuroBench, a benchmark built by the authors: the internal test cohort plus 24 public datasets from 21 countries (including MR-RATE, CTRG, CQ500, RSNA and a pool of 20 brain MRI datasets), for a total of 31,975 closed-ended questions and 3,933 report-writing tasks on 10,481 exams. For datasets that come with reports, the six-option questions were generated by GPT-5 models; for the others, they derive from available diagnostic labels and segmentations. The comparators are GPT-5.4 (through the OpenAI API, "medium" reasoning), Gemma 4 (run locally) and NeuroVFM; 2D models receive the sequence of all slices of each series. Reports are compared pairwise by NeuroJudge, an automatic judge built on the gpt-oss-20B model, whose agreement with three clinical experts was measured on 200 internal exams. Finally, a red-teaming component (adversarial testing) asks trick questions: describe an abscess on a normal exam, or the features of a "glioma" in a patient who had a stroke.

The results

On closed-ended questions, averaged over 17 combinations of tasks and datasets, Neuro-VL beats GPT-5.4 by 15.9 percentage points, Gemma 4 by 16.9 and NeuroVFM by 23.0 (p < 0.001 for each comparison). It beats GPT-5.4 and Gemma 4 on 16 of the 17 groups and NeuroVFM on all 17. On pediatric exams, it reaches 89.2% correct answers, versus 61.8% for GPT-5.4 and 65.1% for Gemma 4. For report writing, NeuroJudge prefers Neuro-VL 2.4 times as often as it prefers GPT-5.4, and 5.0 times as often against Gemma 4.

In longitudinal follow-up, the dedicated variant correctly characterizes the change between two exams in 87.5% of cases (81.6% for GPT-5.4) and the disease trajectory in 96.9% (86.5%). On the public BraTS glioma dataset, the mean absolute error on lesion size is 7 to 9 mm depending on the axis, still too wide to replace a manual measurement when a few millimeters change clinical management.

The most telling result comes from the trick questions. On 320 normal exams, when the question asserts an abnormality and its location ("describe the abscess on the left"), Neuro-VL declines to invent in 97.3% of cases, GPT-5.4 in 51.4%. On 246 glioma or stroke MRIs, when the question puts forward a wrong diagnosis, Neuro-VL stays correct in 90.2% of cases versus 44.5% for GPT-5.4, which was at 99.6% with a neutral question.

In concrete terms: out of 100 normal exams submitted with a leading question, GPT-5.4 would describe a nonexistent lesion about 49 times, Neuro-VL about 3 times. Out of 1,000 closed-ended questions, the average 15.9-point gap means roughly 160 more correct answers. But these figures measure accuracy on multiple-choice questions, not sensitivity or specificity for a given diagnosis, and say nothing about how a radiologist's reading would change with the tool.

What is good

First, the scale and rigor of the split. Nearly one million exams from real care, MRI and CT, over thirty years and several manufacturers, with a temporal and patient-level split: this is well above most published neuroimaging models, and the split limits the risk of data leakage between training and internal testing.

Second, a broad evaluation and a real robustness test. Twenty-four public datasets from 21 countries, pediatric exams, longitudinal follow-up, and above all an adversarial component that measures a model's tendency to follow a false premise, one of the most dangerous failure modes of LLMs in clinical use. Showing that a widely used general-purpose model invents a lesion about half the time when it is suggested is a useful result in itself.

Third, openness. The code (GitHub), the weights of the 8-billion-parameter model (Hugging Face) and the NeuroBench dataset are released under a CC BY-NC-SA 4.0 license; the benchmark code is MIT-licensed. Funding is mainly public and philanthropic (NIH, Chan Zuckerberg Initiative, foundations), and the protocol was approved by the university's institutional review board.

What is less good

First, a circular evaluation chain, a form of misleading metric. Training examples and part of the test questions are generated by GPT-5 models from reports; generated reports are judged by another LLM, gpt-oss-20B; adversarial tests are scored by the same kind of judge. Part of the advantage may therefore come from Neuro-VL's familiarity with the format and style of questions generated the same way. The judge's agreement with experts is only given in a supplementary figure, and the preprint reports no rate of factual errors in reports, only preference ratios. The text also does not say whether any of the NeuroBench public datasets were excluded from training.

Second, a biased comparator. No radiologist took the benchmark and no reader study is reported: a clinician interaction study is described as underway. Against GPT-5.4 and Gemma 4, the comparison pits a specialized volumetric model against 2D models fed long sequences of slices with default settings, a setup that structurally disadvantages them. NeuroVFM, the other specialized comparator, comes from the same laboratory. The preprint claims to outperform frontier models; it says nothing about performance against a neuroradiologist or a cleared hemorrhage-detection tool.

Third, possible population bias and incomplete transparency. All training comes from a single health system in the US Midwest; demographics (age, sex, ancestry) are not described in the main text, subgroup analyses are relegated to supplementary figures, and no analysis by manufacturer or protocol is reported. The evaluation is entirely retrospective, which the authors acknowledge. We did not find a competing-interests statement in the version we read. Finally, the non-commercial license rules out deployment as is, and no regulatory status is claimed.

What it changes

For the research community, Neuro-VL provides an open, volumetric reference point for conversational neuroimaging, with a reusable public benchmark. Above all, it shows that an 8-billion-parameter model trained on one hospital's data can outperform frontier general-purpose models in its own domain. The expected next step is clear: a reader study, rates of clinically significant errors in reports, and validation on non-public data from other health systems.

For clinicians, nothing changes in practice today: the model is neither cleared nor evaluated in real-world conditions, and its performance is expressed as average accuracy on generated questions. The adversarial component, however, carries an immediate message: asking a general-purpose assistant a question that embeds the hypothesis ("describe the hemorrhage") can be enough to make it see what is not there.

For patients and the public, this work illustrates a broader trend: specialized models trained on hospital archives that might one day offer a first reading of a brain scan or track how a tumor evolves. But between a benchmark and a reliable report handed to a patient, the decisive step is still missing: proof, against physicians and in real conditions, that the tool improves reading without adding errors.

Further reading

The preprint: A frontier machine intelligence for human neuroimaging, Zhao, Rao, Kondepudi et al., University of Michigan and University of Tennessee Health Science Center, arXiv (cs.AI), posted 11 November 2026, DOI 10.48550/arXiv.2611.15094. Not peer reviewed.

Code and weights: github.com/MLNeurosurg/neuro-vl and huggingface.co/mlinslab/neuro-vl-8b-thinking (CC BY-NC-SA 4.0). Benchmark: NeuroBench on Hugging Face. The lab's earlier work: Health system learning achieves generalist neuroimaging models (NeuroVFM).

Editorial transparency note: this article was produced by Tatakoto's automated editorial pipeline from version 1 of the preprint (arXiv HTML version); the supplementary figures could not be consulted. The "out of 100 exams" and "out of 1,000 questions" conversions are our calculations from the published percentages. The English, Spanish and Chinese versions are AI-generated. Spotted an error? Contact us.