Turning a network's attention into a biomarker: Grad-CAM-guided "radiomic signatures" to classify tumors — marginal gains, clinical validation barely above chance

A University of Florida team proposes a pipeline meant to make AI tumor classification "interpretable": a network segments the tumor, a Grad-CAM attention map flags its most salient regions, a mutual-information thresholding step carves out a per-patient "signature," and radiomic features are extracted from it, classified by simple models and explained with SHAP. On public breast (BUSI) and brain (BraTS) tumor datasets, adding this signature improves classification — by 0.2 to 3 AUC points depending on the network. But on the only real clinical cohort, UF Health renal CT scans, every deep network stays glued to chance (AUC from 0.49 to 0.59), and the preprint gives no sample sizes, no code, and no funding or conflict-of-interest statement — which hollows out the words "clinical validation" in the title.

The context

Radiomics means extracting from a medical image dozens to hundreds of quantitative variables — lesion shape, pixel-intensity statistics, texture measures — to characterize a tumor reproducibly, beyond the radiologist's eye. For a decade, deep learning (neural networks with many layers) has outperformed classical radiomics on many classification tasks, but at the cost of opacity: you get a prediction without knowing what the network relied on. This "black box" is a real barrier to clinical adoption, where the practitioner wants to understand why a case is labeled malignant.

A common answer is Grad-CAM (Gradient-weighted Class Activation Mapping): a post-hoc method that colors the image regions that weighed most in the network's decision — a heatmap of the model's "attention." The paper's bet is to go one step further: turn that attention map into a reproducible imaging biomarker, i.e., cut out a precise region of the tumor, extract interpretable radiomic variables from it, and check they suffice to classify. The stated ambition is twofold: signature "discovery" and "clinical validation." It is this second promise that deserves close scrutiny.

The method

The pipeline chains four steps. First, segmentation: two reference networks, nnU-Net and UNETR, delineate the tumor, and the union of their two masks is kept. Then Grad-CAM is applied to the final decoder layer to produce a saliency map. That map undergoes an exponential decay with distance from the center (the coefficient adjusts to lesion radius: the smaller the lesion, the steeper the decay). Then comes the cleverest piece: a mutual-information thresholding. Mutual information measures how much two maps "resemble" each other in shared information; here, all 256 possible thresholds of the attention map are tested and the one whose binarized region shares the most information with the segmentation mask is kept. This yields a per-patient "signature" — the portion of tumor that is both salient to the network and consistent with the segmented anatomy.

From this region, two paths branch. The first extracts, via the PyRadiomics library, classical radiomic variables (first-order statistics, shape, and GLCM, GLRLM, GLSZM textures), fed to four simple models — random forest, XGBoost, linear SVM, logistic regression — whose decisions are then explained with SHAP (a method assigning each variable its contribution to the score). The second path trains four classification networks directly (DenseNet121, ResNet50, EfficientNet, ViT) in three input settings: image only, image + mask, image + mask + signature — to measure what the signature adds.

Four datasets are used. BUSI, breast ultrasound, for benign/malignant classification. BraTS, multimodal brain MRI (four sequences), to distinguish glioma from meningioma. KiTS, renal CT, used only to train segmentation. And a private UF Health cohort, renal cell carcinoma CT, for benign/malignant classification — the only real clinical data, not from a public benchmark. Evaluation rests on repeated 80/20 splits with different random seeds, reported as mean and standard deviation. Crucially for what follows: the preprint reports no sample sizes — no patient count, no image count, no class sizes, for any of the four datasets.

The results

On the public datasets, adding the signature helps, but modestly. In deep classification, the image + mask + signature setting is best in 11 of 12 model × dataset combinations. On BraTS, DenseNet reaches the best AUC, 0.960 (F1 = 0.866); the signature's gains range from 0.2 to 3 AUC points depending on the network. On BUSI, DenseNet climbs to 0.883, and the most spectacular case is the ViT, going from 0.609 (image only) to 0.851 (with the mask) then 0.857 (with the signature) — most of the gain coming from the mask, not the signature. For classical radiomics, AUCs peak at 0.947 on BraTS (logistic regression) and 0.898 on BUSI (XGBoost), the signature adding 0.6 to nearly 11 points depending on the classifier.

Then comes the clinical cohort, and the picture flips. On UF Health renal CT, every deep network stays near chance: the best AUC is 0.594 (ResNet, image + mask), and most settings hover around 0.49–0.59. The signature changes almost nothing (EfficientNet goes from 0.491 to 0.567; ViT from 0.546 to 0.588; ResNet even drops slightly). Clinical translation: an AUC of 0.5 means that, drawing at random a benign and a malignant tumor, the model gives the higher score to the right one one time in two — the performance of a coin toss. The simple radiomic models do a little better on this cohort (up to 0.730 with logistic regression), but that very contrast — modest radiomics staying afloat, deep learning at chance — mostly signals that nothing here is stable. Since the paper provides no sample sizes, one cannot even convert these AUCs into a count of false positives and false negatives per 1,000 patients.

What is good

A real problem, and an elegant technical idea. The interpretability of tumor classifiers is a genuine barrier to clinical use, and the paper tackles it head-on. The mutual-information thresholding, which picks for each patient the cut of the attention map most consistent with the segmented anatomy, is a careful and reusable mechanism — a clean way to go from a fuzzy heatmap to a measurable region.

A multi-setting evaluation, honest about the bad numbers. Testing four deep networks and four radiomic models, across three tasks, in three input settings, with mean and standard deviation over repeated splits, gives a nuanced view of the signature's real contribution. Above all, the authors publish the catastrophic renal-cohort results instead of hiding them — the AUC of 0.49 sits in black and white in the main table.

An explicit caveat about the nature of Grad-CAM. The text acknowledges that Grad-CAM "reflects model attention rather than true causal pathology" and that highlighted regions "should be interpreted with caution in clinical contexts." This warning, rare in a literature that sells "explainable AI" as proof, is to the authors' credit.

What is less good

The title's "clinical validation" does not hold — this is the central point. The good numbers all come from carefully curated public benchmarks (BUSI, BraTS); the only real clinical data, the UF Health renal cohort, collapses to chance. This is a textbook population bias: a model that shines on clean, curated sets but does not generalize to everyday hospital scans. Titling something "Clinical Validation" when the clinical result is AUC 0.5 is precisely the kind of misleading metric — here, a benchmark success screening a real-data failure — that critical reading must expose.

Elevating Grad-CAM attention into a "biomarker" invites shortcut learning. Grad-CAM maps are notoriously unstable and may highlight an acquisition artifact, an edge, or a device marker rather than the pathology. Thresholding them by mutual information produces a rigorous-looking region, but nothing guarantees it captures tumor biology rather than a spurious correlation. And the gains are thin (often 0.2 to 3 points, within the reported standard deviations): one thus certifies as a "signature" something whose contribution is small and possibly incidental — and the failure on the renal cohort suggests precisely that what the network "looks at" does not transfer from one dataset to another.

A transparency and reproducibility deficit hard to accept for a "clinical" paper. No sample size is given (impossible to know how many patients anything rests on), no code link, no data availability, no license, no funding or conflict-of-interest statement, and — while the cohort uses real patient scans — no mention of ethics approval. Anti-data-leakage controls are not described: we only know that one subset serves segmentation and another classification, with no explicit guarantee that a given patient does not appear on both sides. The only comparator is "whole-tumor" radiomics; no confrontation with an established radiomics pipeline or a clinical baseline. Note finally, in the affiliation list, a co-author attached to TikTok — unusual for a medical-imaging work, and which makes the missing conflict-of-interest statement all the more regrettable.

What it changes

For the research community, the net contribution is a reusable recipe to convert a saliency map into a radiomic region of interest, with an adaptive thresholding mechanism worth borrowing and testing. But the paper's real lesson is unintentional: the yawning gap between public benchmarks (AUC up to 0.96) and a real hospital cohort (AUC ≈ 0.5) illustrates, once more, that performance on curated data says almost nothing about clinical performance. The credible next steps are known: publish sample sizes, guarantee patient-level splits, validate externally and multicenter, compare to standard radiomics, and above all study the stability of Grad-CAM maps before calling them "biomarkers."

For clinicians, there is nothing deployable: on the only clinical task tested, the tool does no better than chance, and the whole remains an unapproved research prototype. For patients and the general public, the message is one of reading hygiene: the "interpretable AI" label does not mean "AI that works." A pretty heatmap colored over a tumor is not proof the model is right — it shows where it looked, not whether it saw correctly. And a flattering number on a demonstration dataset says nothing about behavior on the images of a real radiology department.

To go further

The preprint is available on arXiv (10.48550/arXiv.2607.03593). On the trustworthiness and fusion of representations in multimodal cancer analysis, see our decryptage on foundation models in oncology and their trustworthiness; on what SHAP-explainable scores are really worth, that of an SVM interpreted by SHAP in oncology; and on deep classification of brain tumors and its pitfalls, that of a TDA-ViT model on brain MRI.

Editorial transparency: French version written and signed by the Tatakoto editorial team based on a reading of the preprint. English, Spanish and Chinese translations produced with AI assistance and reviewed.