A Multimodal Deep Learning Approach for Atrial Fibrillation Classification from 12-Lead ECG Recordings

• Rafael Laranjeira UFAL
• Bruno Pimentel UFAL
• Thiago Cordeiro UFAL
• Álvaro Sobrinho UFAL
• Estela Ribeiro UFAL
• Felipe Dias UFAL
• Marco Antonio Gutierrez UFAL

DOI: https://doi.org/10.5753/sbcas.2026.21745

Resumo

This paper presents a multimodal approach for the automated classification of Atrial Fibrillation (AF). Our approach converts standard 12-lead Electrocardiogram (ECG) images into complementary modalities (i.e., images, time series, and spectrograms) to support neural network analyses. A weighted fusion mechanism combines modality-specific features and adapts their contributions to different classification challenges. We evaluated the approach under a balanced class distribution by downsampling the majority class (Normal Rhythm) to match the number of samples in the minority class (AF). Using the InCor-DB private dataset, the implemented multimodal fusion models achieved an F1 score of 99.28% in the balanced scenario and 88.59% in the imbalanced one. Additional validation on the Zheng-DB public dataset confirmed model generalization, with an F1 Score of 98.13% under balanced conditions. Our results demonstrate the feasibility of combining spectrograms, images, and time series for automated AF classification.

Referências

Aldughayfiq, B., Ashfaq, F., Jhanjhi, N., and Humayun, M. (2023). A deep learning approach for atrial fibrillation classification using multi-feature time series data from ecg and ppg. Diagnostics, 13(14):2442.

Chousou, P. A., Chattopadhyay, R., Tsampasian, V., Vassiliou, V. S., and Pugh, P. J. (2023). Electrocardiographic predictors of atrial fibrillation. Medical Sciences, 11(2):30.

Dias, F. M., Ribeiro, E., Moreno, R. A., Ribeiro, A. H., Samesima, N., Pastore, C. A., Krieger, J. E., and Gutierrez, M. A. (2023). Artificial intelligence-driven screening system for rapid image-based classification of 12-lead ecg exams: A promising solution for emergency room prioritization. Ieee Access.

Geldsetzer, P. and Tisdale (2024). The prevalence of cardiovascular disease risk factors among adults living in extreme poverty. Nature Human Behaviour, 8:903–916.

Jeon, H., Jung, Y., Lee, S., and Jung, Y. (2020). Area-efficient short-time fourier transform processor for time–frequency analysis of non-stationary signals. Applied Sciences, 10(20):7208.

Nesheiwat, Z., Goyal, A., and Jagtap, M. (2024). Atrial Fibrillation. StatPearls Publishing, Treasure Island (FL), Updated 2023 Apr 26 edition. StatPearls [Internet].

Ping, Y., Chen, C., Wu, L., Wang, Y., and Shu, M. (2020). Automatic detection of atrial fibrillation based on cnn-lstm and shortcut connection. In Healthcare, volume 8, page 139. MDPI.

Zheng, J., Zhang, J., Danioko, S., Yao, H., Guo, H., and Rakovski, C. (2020). A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients. Scientific data, 7(1):48.

Zihlmann, M., Perekrestenko, D., and Tschannen, M. (2017). Convolutional recurrent neural networks for electrocardiogram classification. In 2017 Computing in Cardiology (CinC), pages 1–4. IEEE.