Pulmonary embolism: two commercial Aidoc algorithms evaluated across 17 sites, with 86.8% sensitivity falling to 72.9% for small emboli
A team at Emory University (with Yale) measured the performance of two commercial Aidoc algorithms that flag pulmonary embolism on CT, using two cohorts of 30,678 and 37,191 examinations from 17 facilities of a single health system. Sensitivity is 86.8% for embolism on dedicated CT angiography and 73.5% for incidentally found embolism, with specificity above 99% in both cases. The result is useful because it is independent of the manufacturer and exposes a gap with the FDA clearance benchmarks, but it rests on a reference extracted from radiology reports by an LLM, in a single health system, and without comparison to a human read.
The context
Pulmonary embolism (PE) is the blockage of a lung artery by a clot, most often coming from a deep leg vein. It is confirmed by CT pulmonary angiography (CTPA, a contrast-enhanced CT focused on the pulmonary arteries). Delayed reading has clinical consequences, hence the interest in "triage" software: the algorithm analyses each examination as soon as it is acquired and, if it judges it suspicious, moves the case to the top of the radiologist's worklist. It does not make the diagnosis; it changes the order of reading.
Besides PE that is being looked for, there is incidental PE (iPE): a clot found by chance on a CT ordered for another reason, for example a cancer work-up. These cases are easy to miss because the radiologist is not looking for that lesion. Dedicated algorithms exist for this situation too.
This software is cleared by the US FDA as software as a medical device, on the basis of studies supplied by the manufacturer. Independent post-market evaluations on large volumes of routine examinations remain rare. That is the niche of this paper, published in npj Digital Medicine on 21 October 2026; the publisher's page states that it is an "early share" version that may be edited before the final version of record.
The method
The study is retrospective: it covers examinations that had already been performed. It evaluates two algorithms from Aidoc Medical's BriefCase platform, both FDA-cleared: one for PE triage on dedicated CT angiography (30,678 examinations), the other for iPE detection on routine contrast-enhanced CTs (37,191 examinations). The examinations come from 17 facilities of one academic health system.
The key question for such an evaluation is the "ground truth", meaning what serves as the reference to say whether the embolism was really there. Here, labels were extracted from radiology reports by an LLM-based pipeline (large language model, a language model that reads and interprets text). The authors report that this pipeline was validated, with 97% accuracy and a kappa of 0.94 (kappa measures agreement between two raters after correcting for chance; 1 is perfect agreement). Aidoc supplied aggregated outputs of its algorithms and, according to the authors, had no role in design, analysis, interpretation or writing.
The metrics are sensitivity (the share of real embolisms the algorithm flags) and specificity (the share of examinations without embolism it correctly lets through), along with breakdowns by type of embolism.
The results
ML metrics. For PE on dedicated CT angiography, sensitivity is 86.8% and specificity 99.1%. Sensitivity varies strongly with the type of embolism: 99.3% for saddle emboli (large clots straddling the bifurcation of the pulmonary arteries), but 72.9% for subsegmental emboli (in the smallest branches). It is 89.7% for acute PE and 65.3% for non-acute PE (older, already organised clots). For incidental PE, sensitivity is 73.5% and specificity 99.8%. The authors write that both models had lower sensitivity than the FDA clearance benchmarks and higher specificity.
Clinical translation. Without knowing prevalence, we can already reason in groups of cases: out of 100 examinations that really contain a PE, the first algorithm misses about 13 (100 − 86.8); out of 100 examinations containing an incidental PE, the second misses about 26 (100 − 73.5). In the other direction, out of 100 examinations without PE, the first produces on average fewer than one false alert (0.9) and the second produces 0.2. What these figures do not tell us is the positive predictive value, the probability that an alert is true: it depends on how frequent PE is in the population, which the version consulted does not give. These conversions are our own calculation, not the authors'.
What is good
Real volume, across 17 sites. Nearly 68,000 routine examinations spread over 17 facilities provide an evaluation base much closer to practice than the small, selected datasets of clearance studies.
A breakdown that shows where the algorithm falters. Moving from an overall sensitivity of 86.8% to a range of 99.3% to 72.9%, then to 89.7% versus 65.3% depending on acuity, is the useful information. The average hid that small peripheral clots and old clots are where the tool lets the most through.
Declared independence and a publicly stated unfavourable result. The authors declare no specific funding and no competing interests, state that the manufacturer only supplied aggregated outputs, and publish a sensitivity gap relative to the clearance benchmarks. That is not the usual reflex of industry-funded evaluations.
What is less good
A fragile and possibly circular reference (misleading metric and labels). Ground truth comes from reports, themselves read by an LLM. An embolism the radiologist did not see is in no report: it would be counted as a false alert by the algorithm, which underestimates specificity and may also hide true positives. Conversely, if the algorithm's alert was visible to radiologists during the period, it may have influenced the report (automation bias), which would artificially bring the reference closer to the tool's output. The version read does not say whether this applies, nor whether labels were rechecked against images on a sample.
One health system, one vendor, a retrospective design (population bias). Seventeen sites of the same network probably share acquisition protocols, scanners and population. Nothing says the performance transfers to other centres. The study covers a single manufacturer and measures neither the gain in reading time nor the effect on management or patient outcomes, which are the very justification for a triage tool.
A missing comparator, unreported uncertainty. The authors attribute the weaker performance on peripheral and non-acute emboli to limitations that also affect human readers, but the study as presented does not compare the algorithm with a radiologist read: the claim is therefore not demonstrated here. In addition, the version consulted gives no confidence intervals, no prevalence, no per-site results, no detailed definition of acute and non-acute PE, no statement on data or code availability, and has no limitations section.
What it changes
For the research community. The paper illustrates a large-scale post-market evaluation format that a hospital can run without the manufacturer's help. The ground truth still needs strengthening (image review, not only reports) and comparison with human reading.
For clinicians. A triage tool does not replace reading the whole examination, and the absence of an alert does not mean the absence of embolism, particularly for small subsegmental emboli and older PE, where the measured sensitivity is lowest. The real performance of a tool in a department depends on that hospital; this study does not validate its use in any particular case.
For patients and the public. This software appears well tuned to avoid false alarms and less so to catch every subtle case. The paper does not show that patients gain a measurable benefit in delay or outcomes; that evidence remains to be produced.
Going further
Article: Mansuri A., Chavoshi M., Dapamede T., Bala W., Brown-Mulry B., Isaac R. S., Li H., Li F., Moon J. T., Cohen-Addad D. I. G., Salastekar N. V., Newsome J., Gichoya J. W., Khosravi B., Robichaux C., Trivedi H., "Multi-site real-world performance of commercial AI for pulmonary and incidental pulmonary embolism detection", npj Digital Medicine, 21 October 2026, DOI 10.1038/s41746-026-03418-3 (nature.com/articles/s41746-026-03418-3), open access. Affiliations: Emory University School of Medicine (Radiology and Imaging Sciences; Biomedical Informatics) and Yale University. Regulatory framework: triage software falls under FDA software as a medical device (SaMD).
Editorial transparency note: this article was produced by Tatakoto's automated editorial pipeline from the publisher's page (the "early share" version); the full text and supplementary material could not be consulted. Figures are those of that page; clinical conversions are our own calculation. The English, Spanish and Chinese versions are AI-generated. Found an error? Contact us.