A comprehensive framework for atrial fibrillation classification : from ecg images to multimodal analysis

Abstract

This research presents a comprehensive framework for automated atrial fibrillation (AF) classification that bridges the gap between clinical practice and advanced machine learn ing techniques. We introduce a pipeline that transforms standard 12-lead electrocardiogram (ECG) examination images into multiple complementary representations, enabling a system atic investigation of different approaches to AF classification. The framework processes raw ECG images to extract Lead II data, which is then transformed into three distinct modali ties: processed images, time series, and spectrograms. Each modality is then analyzed using specialized neural network architectures that are optimized for their specific characteristics. The investigation involves two experimental scenarios: a balanced comparison be tween AF and normal rhythms and a clinically realistic scenario that maintains the natural class distributions (AF and non-AF), using a private dataset (InCor-DB) and a public dataset (Zheng-DB). In the balanced scenario, the multimodal approach achieved an F1- score of 99.28% ± 0.02%, while individual modalities consistently reached values above 98.36% ± 0.17%. In the clinically realistic scenario, where AF cases accounted for 8.45% of the data, the robustness of the framework was demonstrated, with the multimodal ap proach achieving an F1-score of 88.59%. External validation using the Zheng-DB dataset confirmed the generalization of the framework, with the multimodal approach maintaining strong performance (F1-score of 98.13 ± 0.36%) under balanced conditions. An important part of our approach is the weighted fusion mechanism that combines fea tures from each modality, using learned weights to determine how different representations contribute to the final analysis. Our experiments show that these weights adapt accord ingly to the task complexity, maintaining balanced contributions across modalities (image: 0.3382±0.0025, spectrogram: 0.3450±0.0033, time series: 0.3167±0.0045) to discriminate AF from normal rhythms, while showing strong specialization (time series: 0.5025±0.1252) to discriminate AF from various arrhythmias. This adaptive behavior demonstrates the mech anism’s ability to optimize feature usage based on specific classification challenges, con tributing to robust performance in different clinical scenarios. This research contributes to the field of automated ECG analysis by providing empirical evidence for the effectiveness of different data representations in AF detection. It was also observed that it is not very common to find studies that address spectrograms, images and time series simultaneously. This study seeks to show the feasibility of combining these inputs and the relationship between them. The framework’s ability to process standard ECG images makes it compatible with environments where only the image format is available, which could facilitate its adoption in environments with fewer resources and doctors available.