médical IA

Hierarchical mixture-of-experts for scleroderma-associated interstitial lung disease: adding the patient record to the CT buys 0.0104 AUC points, at p = 0.804

Published on September 1, 2026 · 13 min read

Eleven authors from Northwestern University assemble a frozen imaging expert — five sub-networks, one per lung lobe — and a patient-record branch split into clinical groups, linked by two stages of learned gates, to detect interstitial lung disease in 597 patients with systemic sclerosis. Mean AUC reaches 0.8750 against 0.8646 for the same architecture stripped of the patient record: a gap of 0.0104 points that the authors test and report as not significant (p = 0.804). The paper is worth less for its performance than for what it reports honestly and for what its gates reveal.

The context

A mixture-of-experts (MoE) is an architecture in which several specialized sub-networks — the experts — coexist, and a small additional network, the gate, decides for each input what share of the decision each one gets. Large language models popularized it for a cost reason: activating only two experts out of sixty-four gives you many parameters without paying for much compute. In medical imaging the argument is different. What you want is not compute savings but specialization by anatomical region, plus the interpretability the gate weights provide: knowing which part of the lung, or which family of tests, actually carried the decision.

Interstitial lung disease (ILD) refers to a group of diseases affecting the supporting tissue between the lung's air sacs, thickening and scarring it and impeding oxygen transfer into the blood. In patients with systemic sclerosis — an autoimmune disease that hardens the skin and fibroses internal organs — ILD has become the leading cause of death. Diagnosis rests on high-resolution chest CT read region by region: what matters is not the mere presence of abnormalities but their distribution, with a characteristic basal and subpleural predominance. Yet most deep networks treat the lung as a single volume, diluting precisely the regional signal the radiologist is looking for.

The same team published REN in 2025, an imaging-only MoE structured by lobe that exploited this anatomical intuition. The present work adds the structured electronic health record (EHR): pulmonary function tests, autoantibodies, laboratory values, vital signs. The question it poses is the one the whole multimodal fusion literature in health poses: when two information sources have diagnostic value that varies from patient to patient, should you concatenate them, let them talk through cross-attention, or route them explicitly?

The method

The cohort. Retrospective, from the Northwestern Scleroderma Registry: 597 patients and 1,898 longitudinal chest CT scans acquired between 2001 and 2023. 489 female patients (81.9 %), mean age 63.7 ± 12.7 years. Subtypes: limited cutaneous 47.6 %, diffuse cutaneous 41.0 %, sine scleroderma 4.7 %, other 6.7 %. ILD was confirmed in 365 patients (61.1 %), forming the positive class. Approved by the Northwestern IRB with informed consent.

Preprocessing. Volumes are resampled to 1 mm isotropic spacing, segmented into five lung lobes (left upper and lower, right upper, middle and lower) using LungMask R231, intensity-windowed between −175 and 250 Hounsfield units, then resized to 96 × 96 × 96 voxels.

Radiomics-derived lobe weights. Before training the network, the authors estimate how much diagnostic importance to give each lobe. For each cross-validation fold they extract 107 PyRadiomics features per lobe — texture, shape and intensity measurements computed on the image — and train one XGBoost classifier per lobe on the training partition only. The resulting validation AUC is converted into a weight by a lower-bounded affine mapping: w = 0.1 + (AUC − 0.5) × 3.8 when AUC exceeds 0.5, and 0.1 otherwise. These weights are recomputed for every fold, then held fixed throughout multimodal training.

The imaging expert. Five specialized experts, one per lobe, each a 3D SwinUNETR — a shifted-window transformer designed for medical volumes — applied to the lobe-masked volume. Their representations are aggregated using the normalized radiomics weights, then reduced to a 48-dimensional vector. This expert is trained alone on CT, then frozen: during the multimodal phase its parameters no longer move.

The patient-record branch. For each scan, every clinical variable is linked to the most recent measurement on or before the scan date; failing that, to the nearest subsequent measurement within a 30-day grace window; otherwise the value is zero-filled after standardization. Standardization is fitted on the training partition of each fold only. Two configurations are compared. The full hierarchy splits 69 variables into seven clinical groups: complete blood count (21), pulmonary function tests (3), serum chemistries (14), laboratory and functional markers (4), vital signs (7), static demographics (14) and disease-specific demographics (6). The selective hierarchy keeps only 12 variables in three groups: pulmonary function (FVC, FEV1, DLCO), disease phenotype (lcSSc, dcSSc and sine scleroderma subtypes, plus Scl-70, anticentromere and RNA polymerase III antibody status) and key biomarkers (peak creatinine, modified Rodnan skin score, B-type natriuretic peptide).

The two gating stages. Each group passes through its own small network with a residual projection, producing a 24-dimensional vector; a first softmax gate weights the groups against each other and produces the 48-dimensional record representation. A second gate, the modality gate, takes the concatenation of both representations and produces two weights summing to one: the final representation is their convex combination, fed to a lightweight binary classifier. The point of this structure is theoretical as much as practical: grouping variables reduces the dimensionality of cross-interactions from the order of the squared number of variables to the order of the number of groups, which controls variance.

The protocol. AdamW (learning rate 4 × 10⁻⁴, weight decay 10⁻⁵), batch size 4, early stopping with patience 10, inverse-frequency class weighting. Evaluation by strict patient-level five-fold cross-validation: all scans from one patient stay in the same fold. Comparisons use two-sided paired t-tests on fold-level AUCs computed from identical partitions. Implemented in PyTorch and MONAI on A100 GPUs.

The results

The full ranking. Selective hierarchical MoE 0.8750 ± 0.0443 (95 % CI [0.820, 0.930]); imaging-only REN 0.8646 ± 0.0467 ([0.806, 0.923]); concatenation plus logistic regression 0.8595 ± 0.0438; full hierarchical MoE 0.8496 ± 0.0449; concatenation plus MLP 0.8337 ± 0.0551; EHR-only logistic regression 0.8280 ± 0.0517; whole-lung SwinUNETR 0.7685 ± 0.0759; CNN 0.7584 ± 0.0919; Mamba 0.6775 ± 0.0454; ViT 0.6535 ± 0.0356. A reading reminder: an AUC of 0.875 means that if you draw at random one scan from a patient with ILD and one from a patient without, the model orders the pair correctly about 88 times out of 100, against 50 out of 100 by chance.

The main result is negative, and the authors say so. The 0.0104-point gap between the proposed model and the imaging-only architecture it derives from gives t(4) = 0.265, p = 0.804. The confidence intervals overlap over almost their whole length. Against whole-lung SwinUNETR the difference is clear (p < 0.001), but that comparison has no stakes: it mostly measures the contribution of the anatomical decomposition, already established in the earlier work.

The order of the rows is more informative than the top row. A logistic regression on the record variables alone, with no image at all, reaches 0.8280 — most of the signal. A trivial concatenation of the two modalities followed by logistic regression reaches 0.8595, within 0.0155 points of the full two-stage hierarchical model. On this cohort, in other words, the entire routing machinery buys about one and a half hundredths over the simplest thing you could write.

Less patient record beats more. The selective hierarchy (12 variables) beats the full one (69 variables) by 0.0254 points. The gate analysis offers an explanation: in the full configuration routing spreads across seven groups, several of them static or redundant, and the record branch ends up dominating imaging at high confidence — the regime associated with overconfident positive predictions. In the selective configuration routing concentrates on pulmonary function tests, and the balance between modalities stays stable (mean weights: imaging 0.498 ± 0.122, record 0.502 ± 0.122).

Calibration, errors and clinical translation. Expected calibration error is 0.131 ± 0.038 and Brier score 0.135 ± 0.037. Of 960 test samples, 167 are misclassified (17.4 %), including 71 — 42.5 % of errors — at confidence at or above 0.8. Misclassified cases rely less on the record (mean gate weight 0.383 against 0.527 for correct predictions), and false positives are predominantly imaging-driven. The paper reports neither sensitivity nor specificity, so no fine clinical translation is possible; what can be derived is crude but telling: roughly 174 misclassified scans per 1,000, about 74 of them at high confidence. A system that is wrong one time in six and, in two errors out of five, is confidently wrong, cannot sit downstream of a decision without review.

Stability. Among the 192 patients with more than one scan, the within-patient standard deviation of predicted probability is 0.0812 ± 0.0964, with a mean within-patient range of 0.2004 ± 0.2406: from one scan to the next in the same patient the stated probability can move twenty points, while the label is fixed. Varying the temporal grace window between 0 and 90 days changes almost nothing (AUC 0.8746 to 0.8750), which rules out at least one source of artefact.

What is good

The anti-leakage discipline is explicit and complete. Three places where the health-AI literature routinely leaks are plugged at once. All scans from one patient stay in the same fold, which forbids patient-level leakage — the classic trap in longitudinal cohorts, where a model can recognize a lung it has already seen rather than a disease. Radiomics lobe weights are recomputed within each fold on the training partition only, rather than estimated once on the whole cohort. Clinical variable standardization is fitted on the training split and applied unchanged to validation and test. In a field where data leakage is the leading producer of inflated AUCs, this is the paper's strongest point.

The negative result is tested and published. The direct comparison against the previous architecture is made on identical folds, which is the right way to test an incremental improvement, and p = 0.804 appears in the body of the text, not in an appendix. Many papers would have stopped at the "+13.9 % against SwinUNETR" column, which is accurate and uninteresting. The authors write explicitly that the numerical improvement "requires confirmation in a larger evaluation".

The battery of checks goes beyond what is demanded. Sensitivity of the record-to-scan linkage window across six values, within-patient variance across longitudinal series, calibration measured by two metrics, error analysis stratified by gate weight, and a full ladder of naive fusion comparators that lets the reader locate the gain. To which is added a rare precaution: the authors state plainly that gate activations reflect routing behaviour rather than causal importance, and that a high weight does not establish a test's independent clinical importance.

What is less good

The winner is picked inside the cross-validation that measures it, and there is no human comparator. Two record configurations are evaluated on the same five folds, and the better of the two becomes "the model" — selection on the evaluation set, which mechanically yields an optimistic estimate. Moreover, with five folds a paired t-test has almost no power: t(4) detects only large differences. Saying "not significant" is honest, but failing to conclude is not concluding there is no effect — the design settles nothing either way. More fundamentally, the biased comparator here is the absence of a comparator: the standard for diagnosing scleroderma-associated ILD is a radiologist reading a high-resolution CT, possibly in a multidisciplinary meeting. No human performance is reported, so we do not know whether 0.875 is good, mediocre or trivial in this specific setting.

Misleading metric and the wrong unit of analysis. Only AUC is reported: no sensitivity, no specificity, no operating threshold, when deployment depends entirely on the threshold chosen. Prevalence is 61.1 % in this registry — a population already selected on scleroderma and already imaged — which bears no relation to the prevalence at which such a tool would be used elsewhere, and makes the performance non-transferable as stated. Above all, the unit of evaluation is the scan, not the patient: the 1,898 scans produce predictions we have just seen can move twenty points within one patient whose status is nonetheless single and fixed. The authors do list patient-level endpoints among future work. Finally, an ECE of 0.131 is described as "reasonable calibration": thirteen points of average gap between stated probability and observed frequency is a generous label.

Population bias, missing values indistinguishable from the mean, and opacity. Single centre, 81.9 % female, scleroderma only: generalization to other ILD aetiologies — idiopathic pulmonary fibrosis, rheumatoid arthritis, hypersensitivity pneumonitis — is neither tested nor established, and the authors acknowledge it. No external validation. The quietest point is also the most problematic: missing clinical values are zero-filled after standardization, with no missingness indicator. In that space, "value absent" and "value exactly average" are the same number. Yet in medicine, a missing pulmonary function test is never random — it depends on severity, on access to care, on the year of inclusion. The model cannot tell the two situations apart, and the paper flags the limitation without measuring its effect. On top of which no code or weights link, no funding statement and no conflict-of-interest declaration appear in the preprint; the registry is not described as accessible.

What it changes

For the research community. The reusable contribution is not the gain, it is the comparator ladder. Read bottom-up, it says three things. A ViT or a Mamba applied to the whole lung volume does barely better than chance on this task (0.65 and 0.68). Imposing anatomical structure by lobe is worth about ten AUC points (0.7685 → 0.8646). And the whole hierarchical multimodal apparatus then adds one hundredth of a point, not significant. The message for anyone designing such a system is sharp: the returns are in the anatomical prior, not in the sophistication of the fusion. The second lesson concerns the gates themselves — restricting the clinical branch to mechanistically proximal variables beats giving it sixty-nine, because an unconstrained gate ends up over-weighting the record where imaging carries the information.

For clinicians. Nothing changes today, and it is worth seeing why the task itself is questionable. Detecting ILD in a scleroderma patient who has already had a high-resolution CT and pulmonary function tests is not an open clinical problem: it is what the radiology report produces, and the model's two inputs are exactly the two tests already performed. The real need lies elsewhere — identifying who will progress, how fast, and who would benefit from early antifibrotic treatment. The authors point to this in future work. In the meantime the transferable part is the routing analysis: the model leans on imaging to exclude and on the record to confirm, reproducing the clinical approach without having been told to.

For patients and the public. This work produces no tool in clinical or regulatory evaluation, and for now concerns one rare disease in a single American hospital. What it illustrates deserves to be known outside the technical field: in the health-AI literature, the gap between "our model achieves the best AUC" and "our model is better" is often the size of statistical noise, and that difference is only visible by reading the accompanying test — when there is one. Here the authors ran the test and published its result, which cuts against their immediate interest. That should be the default expectation; it remains rare enough to be worth noting.