Fusão de Dados de Vídeos RGB e Pontos-Chaves para Classificação de Movimentos Gerais de Bebês

• Matheus Palheta UFAM
• Giovanna Santos UFAM
• Ayrles Mendonça UFAM
• Paulo Gonçalves UFAM
• Rafael Albuquerque UFAM
• Eduardo Souto UFAM
• Eulanda M. dos Santos UFAM

DOI: https://doi.org/10.5753/sbcas.2023.230047

Abstract

The detection of atypical spontaneous movements of babies, called general movements, can help in the early identification of neurodevelopmental disorders and consequent early intervention. General movements can be evaluated using video-based data by machine learning techniques. In this paper, babies’ movements are classified into typical and atypical using a video dataset collected for this work. Two data channels are generated from each video: the RGB videos and keypoints. The data provided by each channel is used to train two 3D convolutional neural network classification models, one for each channel. The two models are combined using a fusion function. The results show that the fusion of the models achieves higher classification rates when compared to the rates obtained by the models trained with data provided by each channel individually.

References

Aizawa, C. Y. P., Einspieler, C., Genovesi, F. F., Ibidi, S. M., and Hasue, R. H. (2021). The general movement checklist: A guide to the assessment of general movements during preterm and term age. Jornal de Pediatria, 97(J. Pediatr. (Rio J.), 2021 97(4)):445–452.

Chambers, C., Seethapathi, N., Saluja, R., Loeb, H., Pierce, S. R., Bogen, D. K., Prosser, L., Johnson, M. J., and Kording, K. P. (2020). Computer vision to automatically assess infant neuromotor risk. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(11):2431–2442.

Einspieler, C., Peharz, R., and Marschik, P. (2016). Fidgety movements – tiny in appearance, but huge in impact. Jornal de Pediatria (Versão em Português), 92:S64–S70.

Hesse, N., Bodensteiner, C., Arens, M., Hofmann, U. G., Weinberger, R., and Schroeder, A. S. (2018). Computer vision for medical infant motion analysis: State of the art and RGB-D data set. In Computer Vision ECCV 2018 Workshops. Springer International Publishing.

Kondratyuk, D., Yuan, L., Li, Y., Zhang, L., Tan, M., Brown, M., and Gong, B. (2021). Movinets: Mobile video networks for efficient video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16020–16030.

Köpüklü, O. (2021). Video augmentation techniques for deep learning.

Leo, M., Bernava, G. M., Carcagnı̀, P., and Distante, C. (2022). Video-based automatic baby motion analysis for early neurological disorder diagnosis: state of the art and future directions. Sensors, 22(3):866.

Raghuram, K., Orlandi, S., Church, P., Chau, T., Uleryk, E., Pechlivanoglou, P., and Shah, V. (2021). Automated movement recognition to predict motor impairment in high-risk infants: a systematic review of diagnostic test accuracy and meta-analysis. Developmental Medicine & Child Neurology, 63(6):637–648.

Raghuram, K., Orlandi, S., Church, P., Luther, M., Kiss, A., and Shah, V. (2022). Automated movement analysis to predict cerebral palsy in very preterm infants: An ambispective cohort study. Children, 9(6):843.

Tsuji, T., Nakashima, S., Hayashi, H., Soh, Z., Furui, A., Shibanoki, T., Shima, K., and Shimatani, K. (2020). Markerless measurement and evaluation of general movements in infants. Scientific reports, 10(1):1–13.