médical IA

Classifying an emergency ECG from its image: a ConvNeXt ensemble nears cardiologists on 18,519 tracings

Published on June 25, 2026 · 9 min read

A team at the São Paulo Heart Institute trains a deep-learning model to classify an emergency electrocardiogram not from the digital signal, but from its image — a scanned paper tracing, a PDF, or a phone photo. On 18,519 emergency ECGs across 12 categories, labelled by 19 cardiologists, their ConvNeXt ensemble reaches a macro F1 of 0.807, just below the 0.820 of the annotating cardiologists, and beats a commercial algorithm in most categories. The point is not a record score: it is showing that the mass of ECGs that exist only as images can be exploited, and that the true performance ceiling is inter-reader agreement, not the size of the training set.

The context

The electrocardiogram is the front-line test of any cardiac emergency: twelve leads, a few seconds of recording, and sometimes a life-or-death decision at the end — an unfolding heart attack is treated in minutes. Over the past decade, deep learning has made enormous progress on the ECG, but almost always from the digital signal: the raw voltage time series, clean and structured, as stored by a modern device. Yet much of clinical reality looks nothing like that.

In many emergency departments, and even more in resource-limited health systems, the ECG circulates as an image: a tracing printed on graph paper, a scanned report, an archival PDF, or simply a phone photo taken to seek a remote opinion. The original digital signal is often inaccessible — lost, proprietary, or never exported. A whole class of signal-trained models then becomes unusable. The question posed by this preprint from Felipe Meneguitti Dias and colleagues at InCor (Heart Institute, Hospital das Clínicas, University of São Paulo Medical School) is blunt: can an emergency ECG be classified reliably by reading its image alone?

The method

The authors build InCor-EMG, a set of 18,519 expert-adjudicated emergency ECGs. Each tracing is assigned to one of 12 ECG categories (the various rhythm types and electrical abnormalities seen in the emergency setting), and the labels come from a panel of 19 cardiologists. This matters: the "ground truth" is not an objective measurement but an expert consensus, with its zones of firm agreement and its contested cases.

The model's input is not the signal but the image of the tracing. The system is built on ConvNeXt, a modern convolutional neural-network architecture (a family of models specialised in image analysis, here modernised to compete with vision transformers). Rather than a single network, the authors use an ensemble: several models whose predictions are aggregated, a classic technique to gain robustness and smooth out individual errors.

The main metric is the F1-score, which combines in a single number precision (of the tracings flagged for a category, how many rightly so) and recall (of the true cases of a category, how many are found). The macro F1 averages it over the 12 categories with equal weight — a demanding choice, since a rare category counts as much as a common one and cannot be drowned out. Two comparators provide reference points: the annotating cardiologists (the human comparator) and Mortara Veritas, a widely deployed commercial ECG-interpretation algorithm. Finally, to test sturdiness, the authors evaluate the model on scanned and photographed versions of the tracings, on heterogeneous public images from the LITFL educational library, and in a temporal evaluation (ECGs from after the training period).

The results

On the held-out test set, the ConvNeXt ensemble achieves a macro F1 of 0.807 (95% confidence interval: 0.788–0.825). The annotating cardiologists top out at 0.820 (0.805–0.832). In other words, the image model lands about one and a half hundredths below the human reference — and the confidence intervals overlap widely. Against Mortara Veritas, the commercial algorithm, the model achieves a higher F1 in most of the evaluated categories.

The most instructive result is not the raw score but a correlation: per-category performance is tied more to inter-reader agreement than to training sample size. Plainly, the model does well where cardiologists themselves agree, and stumbles where the experts hesitate — not necessarily where examples are scarce. It is an elegant way of showing that the ceiling of such a system is set by the quality and consistency of the human labels, not just by the data fed in.

The robustness evaluations point the right way: the model stays informative on scanned as on photographed ECGs, and the tests on heterogeneous public images and on a later time window remain "supportive", with no collapse. The honest clinical translation: a macro F1 of 0.807 across 12 categories means an overall good but uneven reading from one category to the next — for the most ambiguous or rarest classes, a non-trivial share of tracings will be misfiled. That is enough to rough-sort, to triage, to flag; it does not replace a cardiologist's eye on a myocardial infarction where every minute counts.

What is good

A real, neglected problem taken seriously. Almost all ECG-AI literature assumes the clean digital signal is available. This work starts from the opposite reality — the tracing exists only as an image — which is precisely that of overloaded emergency rooms and health systems without export infrastructure. It is a relevant clinical framing, not a lab demonstration.

A dual comparator, human and commercial. Many papers settle for an isolated score. Here, performance is set against the annotating cardiologists and a commercial algorithm actually in use (Mortara Veritas). Giving both reference points lets the reader place the model without taking the authors' word for it.

Honesty about the true bottleneck. Showing in black and white that performance tracks inter-reader agreement rather than data size is a useful methodological contribution: it shifts the debate from "more data" to "better labels", and pre-emptively tempers promises of easy gains through scaling alone.

What is less good

A single centre, no prospective clinical validation: the population-bias failure mode. All training and test data come from one institute, InCor in São Paulo, with its devices, print formats and population. A model that reads images is especially exposed to domain shift: another hospital prints on different paper, with a different grid, different fonts, different settings. The LITFL public images offer a glimpse of generalisation, but being educational and heterogeneous, they are no substitute for external multicentre validation — and no prospective real-world evaluation is carried out.

A ceiling set by the labels themselves: the misleading-metric failure mode. The human comparator is also the source of the labels: the model is thus measured against the very target used to train it, a form of circularity. The reassuring 0.807 macro F1 aggregates 12 categories of very unequal frequency and can mask gaps on rare or contested classes — precisely those where cardiologist agreement is weak and where an error may matter most. The global figure says nothing about per-category operating thresholds.

The shortcut risk, and partial reproducibility. A model trained on images can learn layout cues — grid type, device brand, lead arrangement — rather than the electrical morphology of the trace (shortcut learning). Nothing proves that is the case here, but the image architecture is intrinsically exposed to it, and the preprint does not fully dispel the doubt. On reproducibility, the InCor-EMG set cannot be released (health data subject to Brazil's LGPD law, CC BY-NC licence): only a few samples, the inference code and the predictions are shared on GitHub. Note finally, with no declared conflict of interest, an industrial funder among the sources (FAPESP, Fundação Zerbini and Foxconn Brazil) — useful context to know.

What it changes

For the research community, the message is twofold. First, classifying the ECG from its image is a credible route, not a fallback: it opens up the vast archives of tracings that exist only on paper or as PDFs. Second, and more deeply, performance hits the wall of expert agreement — investing in more consistent annotation protocols could pay off more than piling on extra examples.

For clinicians, nothing is deployable as is: single-centre, retrospective, not prospectively validated. But the target scenario is sound — a device able to pre-sort a photographed ECG, where no digital signal is available and specialist advice is remote, would have real value in on-call work, telemedicine or under-resourced areas. Provided it is confined to triage and alerting, never autonomous decisions on life-threatening emergencies. For patients and the public, the takeaway is simple and nuanced: a photo of an ECG will increasingly be analysable by a machine to save time, but the final reading remains a physician's job — an automatic classification is neither a diagnosis nor a guarantee.