ALICE: a single pathology foundation model distilled from eight experts — best average rank across 96 tasks, but no prospective validation

A team from Tsinghua Shenzhen, Oxford, and Sun Yat-sen University fuses eight digital-pathology foundation models into a single network through a three-stage "agglomerative distillation" called ALICE. Across 96 tasks over 48 data sources, ALICE achieves the best average rank against specialized models, beating the runner-up by 1.79, 6.39, and 3.04 points depending on task type. But the evaluation is entirely retrospective, the comparators are ALICE's own teacher models, and no data-leakage control is reported even though pretraining and testing both draw on TCGA.

The context

Digital pathology — the analysis of very high-resolution scanned tissue slides — has become one of the most active arenas in medical AI. Over the past two years, foundation models (large networks pre-trained on huge image corpora, then reusable for many tasks) have proliferated: UNI, Virchow, H-optimus, GigaPath, CONCH, TITAN, and others. The problem is that they are not interchangeable. Each was trained with a different objective, data, and scale. Some excel at describing a small tissue tile, others at linking an image to text, others still at reasoning over an entire slide (the whole-slide image, or WSI, which can span several gigapixels). Expertise is thus "fragmented" across several networks: to cover all tasks, a lab must host and run several heavy models in parallel.

ALICE — for Agglomerative Learning via Integrated Computational pathology Embedding — proposes another route, borrowed from general computer vision (the RADIO method): rather than train yet another model on even more data, one distills several existing models into a single one. The paper asks two questions: can the complementary skills of eight models be consolidated into one network, and does that single network match — or even beat — each of its teachers on their own turf?

The method

Knowledge distillation means training a "student" network to reproduce the internal representations of one or more frozen "teacher" networks, without access to their original data. ALICE pushes the idea to eight teachers, in three families. Three vision-only models (UNI-2, 681 million parameters; Virchow-2, 632 million; H-optimus-1, 1.1 billion) that encode tissue tiles. Three vision-language models (CONCH, MUSK, KEEP) that align image and text. Two slide-level models (TITAN, CARE) that aggregate thousands of tiles into one whole-slide representation.

The student is a ViT-H/14 visual transformer (the transformer is the architecture that has dominated AI since 2017; "H/14" denotes its size and the fineness of its tiling) with three sequentially activated modules. Training proceeds in three stages matching the three teacher families. At each stage, the student learns to mimic the relevant teachers' outputs through two main losses: an angular distillation loss, which brings the directions of the student's and teacher's vectors together, and a patch distillation loss, which aligns local spatial representations. The vision module is trained first, then frozen; the multimodal module is grafted onto it; the slide-level module finally works on pre-computed features, without revisiting raw pixels. Notably, the paper never reports ALICE's total parameter count.

Pretraining uses only public data. For tiles, TCGA-12K: 24,985,184 images of 224×224 pixels extracted from roughly 12,000 slides of the Cancer Genome Atlas (TCGA), the large public US database, covering 32 cancer types. For high resolution, 155,604 regions of 8192×8192 pixels, 25,495 from TCGA and 130,109 from the public HISTAI collection. No private cohort enters pretraining. Evaluation covers 21 scenarios (7 per family), 96 downstream tasks, and 48 data sources: region classification and segmentation, visual question answering, image retrieval, and at the slide level, tumor subtyping, biomarker prediction, and survival analysis. Models are compared on identical datasets and splits, each time against "task-matched" models — which, in practice, means ALICE's own teachers.

The results

The main result is expressed as average rank: on each task, models are ranked, then ALICE's position is averaged. In all three settings, ALICE achieves the best average rank among matched models, beating the runner-up by 1.79 points (vision-only), 6.39 points (multimodal), and 3.04 points (slide-level). Against the average of all other models, the gap widens to 3.10, 7.41, and 4.00 points. In weakly supervised whole-slide classification across 19 clinical tasks, ALICE is first on 17, and significantly better than the runner-up on 12. In few-shot learning (from very few examples) across 12 region tasks, its average rank is 1.08.

These figures are strong but call for careful reading, and the paper itself provides the material to qualify them. Rank hides the magnitude of gaps: ranking 1.25 on average rather than 1.5 may cover minimal absolute differences. And where an absolute metric is given, ALICE is not always ahead: in region instance segmentation, it scores 0.599 AP@50 (a detection-quality measure) versus 0.607 for UNI-2 — a slight step back against one of its own teachers.

Clinical translation remains theoretical, and the authors do not claim otherwise. The "clinical" tasks — predicting ER/PR hormone-receptor status in breast cancer, mutations (PBRM1, BAP1, SETD2) in kidney cancer, survival analysis measured by the concordance index or C-index (the probability that the model correctly orders two patients by prognosis) — are all retrospective, run on archived cohorts. None corresponds to prospective use at the bedside. A record average rank is therefore not a validated diagnostic test.

What is good

An elegant, useful engineering idea. Consolidating eight heavy models into one network answers a real field problem: instead of hosting and maintaining several foundation models in parallel, a lab could run just one. The method, imported from general vision, is transferable to other fragmented domains, and the fact that a single student holds its own against its eight teachers on their own tasks is a non-trivial result.

A broad evaluation, honest about its blind spots. 96 tasks, 48 sources, six metric families: the scope of the test bench exceeds most papers in the field. Above all, the authors also report where ALICE loses (instance segmentation below UNI-2), rather than showing only wins. That transparency about defeats is a marker of seriousness.

Concrete openness. The model's code and weights are announced as public (GitHub repository WonderLandxD/ALICE), and all pretraining data come from public sources hosted on Hugging Face (TCGA-12K, TCGA-UT-8K, HISTAI). This makes the work reproducible and reusable, unlike models trained on inaccessible hospital slides.

What is less good

A possible data leakage, never controlled for. This is the most serious limit. ALICE is pre-trained on TCGA, and several test sets are TCGA-derived or TCGA-adjacent; its eight teachers also saw huge, potentially overlapping slide collections. Yet the paper reports no overlap analysis between pretraining slides and evaluation slides. This data leakage — when test data, directly or indirectly, has served for training — mechanically inflates scores. Without documented deduplication, the benchmark figures should be read as upper bounds.

A comparator that is both judge and party, and a flattering metric. ALICE aggregates the knowledge of eight models, then compares itself… to those same eight models taken one by one. Beating, by construction, teachers whose knowledge one has merged is not the same as beating an independent, more recent model. Add an asymmetry in zero-shot (classification without retraining): ALICE averages the probabilities of its three heads aligned to CONCH, MUSK, and KEEP, giving it an ensemble advantage that single-model comparators lack. Finally, choosing average rank as the showcase — a misleading metric here — masks sometimes tiny absolute gaps.

Thin validation and a narrow data base. The authors acknowledge it plainly: ALICE "was evaluated primarily in retrospective settings," and "prospective, multi-institutional" studies remain necessary given real-world variation in staining, scanner, and population. Pretraining rests on TCGA — mostly US academic centers — hence a possible population bias on under-represented cohorts, and it ignores immunohistochemistry and genomics alike. By nature, a distillation also inherits its teachers' blind spots: if all share the same shortcut learning (learning a preparation artifact rather than biology), the student inherits it without correcting it.

What it changes

For the research community, ALICE establishes distillation-consolidation as a credible strategy against the race for size: instead of always training bigger, one can fuse what exists. It also raises an open question — that of diminishing returns and dependence on teacher quality: a student cannot, in principle, durably surpass what its masters already know. The work provides a reusable unified model, provided the weights are indeed released under a usable license, which the paper does not specify.

For clinicians, nothing changes today. This is a research model evaluated on archived slides, with no prospective study or regulatory clearance. Such a network remains a generic backbone, to be adapted and then validated on the target institution's data before any use.

For patients and the general public, caution is warranted on the wording. "Best average rank across 96 tasks" sounds like a coronation, but denotes a laboratory performance on past data, not a proven diagnostic capacity. The gap between the benchmark score and usefulness at the bedside — the crux of medical AI — remains wide open.

To go further

The preprint is available on arXiv (10.48550/arXiv.2607.09526); code and weights are announced on GitHub. Work funded by Chinese public grants (NSFC 82430062 and others); the authors declare no competing interests. On the opposite question — curating the data rather than distilling the models — see our decryptage on MedPMC and the PMC literature; on a pathology foundation model trained on real-world data, that of GigaPath; and on the critical evaluation of foundation-model representations in oncology, our decryptage probing and fusion in oncology.

Editorial transparency: French version written and signed by the Tatakoto editorial team based on a reading of the preprint. English, Spanish and Chinese translations produced with AI assistance and reviewed.