médical IA

Predicting HIV treatment non-adherence with machine learning: what is a "real-world" validation on 192,732 multi-country records worth? (medRxiv, 2026)

Published on June 8, 2026 · 9 min read

A preprint posted on 15 May 2026 on medRxiv validates machine learning models on 192,732 clinical records from several countries, to predict HIV treatment non-adherence and quantify "gaps" in the care pathway. Discrimination, measured by temporal validation on future patients, reaches a 0.772 AUC (95% CI: 0.744–0.802), and the study documents a median 74-day delay between diagnosis and treatment start, with 47.3% of patients exceeding 90 days. It is an honest, useful large-scale demonstration — but a modest AUC, an adherence outcome whose definition stays opaque in the public abstract, and economic modelling with undetailed assumptions invite reading these numbers for what they are.

The context

HIV is treatable, on two conditions: being diagnosed early and taking one's antiretroviral therapy (ART, the drug combination that blocks viral replication) without interruption. Adherence — how regularly a patient actually takes the treatment — is the crux: irregular intake lets the virus rebound, damages immunity and breeds resistance. In resource-limited settings, where most affected people live, two fractures persist: diagnosis comes too late, and patients are lost along the way. UNAIDS frames the goal as the "95-95-95" cascade (95% of people diagnosed, 95% of those on treatment, 95% of those with a suppressed viral load).

Using machine learning to flag patients at risk of disengaging is not new, but most prior work involved small single-center cohorts, often with no validation beyond the training set. This preprint stands out for its ambition of "real-world" validation and its scale: 192,732 records, several countries. That is exactly what makes it worth decrypting — and what calls for caution about what "real-world validation" means here.

The method

The work is a medRxiv preprint (2026.05.15.26353325, posted 15 May 2026, not yet peer-reviewed). It is a retrospective study on routine data: existing records are reused rather than recruiting patients for the occasion. Two goals run in parallel: predicting future non-adherence, and quantifying "care gaps" — breaks in the care pathway (delays, dropouts, late presentations).

On the modelling side, the abstract notably mentions XGBoost, a gradient-boosted decision tree algorithm: it combines hundreds of small trees, each correcting the previous one's errors, and remains the reference tool for tabular data (columns of variables such as age, CD4 count, visit dates). The claimed validation is temporal: the model is tested on future patients, posterior to the training period, rather than on a sample drawn at random from the same set. That is the right way to simulate a real deployment, where one always predicts on patients not yet seen.

Several decisive elements are absent from the public abstract and will have to be checked in the full text: which countries exactly, how many sites, how "non-adherence" is operationally defined (pharmacy refill measure? viral load? missed appointments?), which variables feed the model, and whether code and data are available. These blind spots condition the interpretation of the results.

The results

On temporal validation, the model reaches a 0.772 AUC (95% CI: 0.744–0.802). The AUC (area under the ROC curve) measures the ability to correctly order two randomly drawn patients, one who will disengage and one who will not: 0.772 means they are ranked in the right order roughly three times out of four. That is moderate discrimination — honest, but far from the AUCs of 0.95 or more announced elsewhere, which often collapse at the first serious validation.

The study also shows reassuring stability across subgroups: the AUC varies little by sex, age, CD4 level and WHO stage, with a maximum gap of 0.051. Above all, the "care gaps" part delivers valuable descriptive figures: a median 74-day delay between diagnosis and treatment start, 47.3% of patients beyond 90 days, and 36.7% presenting with fewer than 200 CD4 per microliter — that is, at a stage of advanced immunodeficiency. An economic model finally projects a base-case saving of USD 415 per patient, i.e. USD 2.07 million for a 5,000-patient cohort.

Clinical translation. A 0.772 AUC can help prioritize — for instance deciding who to call back first — but not decide individually: at this level of discrimination, many at-risk patients will be missed and many labeled "at risk" will not disengage. The abstract gives neither sensitivity, nor specificity, nor positive predictive value at a given threshold; one therefore cannot convert the AUC into a number of false positives and false negatives per 1,000 patients, which would be the only way to judge operational usefulness. The most telling figure is not the AUC: it is the 74-day delay and the 36.7% diagnosed too late.

What is good

The scale and the genuinely multi-country nature of the data. 192,732 routine clinical records, from several countries, is a rare order of magnitude in the ML-HIV literature, long dominated by single-center cohorts of a few thousand patients. Working on field data, with its noise and gaps, is more demanding and more relevant than a clean, homogeneous dataset.

A temporal validation, not a mere random split. Testing on future patients is methodologically the right discipline: it most closely approximates a deployment, and it is exactly the control missing from many studies with spectacular AUCs. That performance holds over time (0.772, tight CI) is more credible than a perfect score on a random split. We drew the same distinction about a thromboembolism risk model in our SHAP-SVM decryption.

Honesty about the numbers and an equity analysis. Reporting a modest 0.772 AUC rather than a too-good 0.99, checking stability across subgroups (maximum 0.051 gap across sexes, ages, CD4 and WHO stages), and above all quantifying the real "care gap" (74 days, 47.3%, 36.7%): the study documents a health-system problem instead of overselling an algorithm. That is the most useful spirit one can hope for in such work.

What is less good

An opaque adherence outcome — risk of a misleading metric. The public abstract does not say how "non-adherence" is defined or measured. Yet everything depends on that definition: if it relies on a variable correlated with the outcome but recorded after the fact, one risks information leakage (the model "cheats" by using a signal that would not be available at prediction time). Without the exact definition, without calibration, without sensitivity/specificity, the 0.772 AUC cannot be translated into concrete clinical usefulness.

"Multi-country" without named countries — population bias not assessable. We do not know which countries the 192,732 records come from, so we cannot judge the model's representativeness or its transportability from one health system to another. This is the classic population bias: a model trained on certain contexts may collapse elsewhere, and temporal validation does not protect against this geographic risk. No prospective validation (forward, under real care conditions) is reported — only a reanalysis of past data.

Fragile projected economics, an unreviewed preprint. The "USD 415 per patient" and "2.07 million for 5,000 patients" rest on a model whose assumptions are not detailed in the abstract: cost of an avoided break, supposed effectiveness of the model-triggered intervention, adoption rate. Such a figure, attractive to a decision-maker, must be handled cautiously until the assumptions are auditable. Add the preprint status: no peer review yet, and code and data availability not established at this stage.

What it changes

For the research community. The work pushes in the right direction: validating on future patients at scale rather than optimizing a clean-room AUC. The most durable contribution is probably not the predictive model itself but the care-gap quantification on nearly 200,000 records — empirical material for anyone seeking to target breaks in the pathway. It remains to publish the adherence definition, the countries, and the code to make all this verifiable.

For clinicians and programs. A model at 0.772 AUC can, at best, help prioritize recalls and close follow-up where resources are scarce — not decide a given patient's fate. But the most actionable message is not algorithmic: it is that nearly one patient in two waits more than 90 days to start treatment, and a third arrive already in advanced immunodeficiency. These are organizational levers, not AI ones.

For patients and the public. Here, for once, is the opposite of an inflated announcement: an AI with modest but honest performance, useful above all to measure a system problem. The right reflex stays the same as with stellar scores — look at what the figure means: here, a projected saving of millions of dollars hinges on assumptions we cannot see, and it is those we will need to revisit. We already stressed the gap between data and clinical reality in our decryption of clinical vision-language models.