Analysis of a Video-Based Pain Monitoring System in Raspberry Pi
Resumo
This work presents an analysis of the efficiency and effectiveness of a Video-Based Pain Monitoring System running on a Raspberry selected because it is a cheap device that can be easily carried around. The objective of the evaluated system is to allow the assessment of pain based on two characteristics: Heart Rate (HR) and facial expressions detected through the Facial Action Coding System (FACS). To measure HR an Eulerian Video Magnification (EVM) based method was implemented. EVM is one of the main current approaches to measure HR by Remote PhotoPlethysmoGraphy. FACS was used to detect pain intensity with the Prkachin and Solomon Pain Intensity (PSPI) equation which is one of the most used approaches to detect pain intensity based on facial features. To identify the PSPI value we trained a Regression Neural Network (RNN) with the UNBC-McMaster database. The experimental results demonstrate the strengths and limitations of the evaluated system.
Palavras-chave:
Action Untis, Remote PhotoPlethysmoGraphy, Paint estimation, Low-cost system
Referências
Thefreedictionary, “Online dictionary,” https://www.thefreedictionary.com, 2021, [Online; accessed 15-Sep-2021].
M. Benjamin and K. CB, “Miller-keane encyclopedia and dictionary of medicine,” Nursing and Allied Health. Philadelphia: Saunders, 1997.
S. N. Raja, D. B. Carr, M. Cohen, N. B. Finnerup, H. Flor, S. Gibson, F. J. Keefe, J. S. Mogil, M. Ringkamp, K. A. Sluka et al., “The revised international association for the study of pain definition of pain: concepts, challenges, and compromises,” Pain, vol. 161, no. 9, pp. 1976- 1982, 2020.
N. Katz, “The impact of pain management on quality of life,” Journal of pain and symptom management, vol. 24, no. 1, pp. S38-S47, 2002.
A. B. Amspoker, A. L. Snow, B. N. Renn, P. Block, S. Pickens, R. O. Morgan, and M. E. Kunik, “Patient versus informal caregiver proxy reports of pain interference in persons with dementia,” Journal of Applied Gerontology, vol. 40, no. 4, pp. 414-422, 2021.
L. P. Carlini, T. M. Heideirich, R. C. Balda, M. C. Barros, R. Guinsburg, and C. E. Thomaz, “Visual perception of pain in neonatal face images,” in Workshop de Visão Computacional. SBC, 2019, pp. 37-42.
L. Buzuti, T. Heideirich, M. Barros, R. Guinsburg, and C. Thomaz, “Neonatal pain assessment from facial expression using deep neural networks,” in Workshop de Visão Computacional, 2020, pp. 87-92.
K. Herr, P. J. Coyne, E. Ely, C. Gélinas, and R. C. Manworren, “Pain assessment in the patient unable to self-report: clinical practice recommendations in support of the aspmn 2019 position statement,” Pain Management Nursing, vol. 20, no. 5, pp. 404-417, 2019.
J. O. Egede, S. Song, T. A. Olugbade, C. Wang, C. D. C. Amanda, H. Meng, M. Aung, N. D. Lane, M. Valstar, and N. Bianchi-Berthouze, “Emopain challenge 2020: Multimodal pain evaluation from facial and bodily expressions,” in IEEE International Conference on Automatic Face and Gesture Recognition, 2020, pp. 849-856.
D. Lopez-Martinez and R. Picard, “Multi-task neural networks for personalized pain recognition from physiological signals,” in International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, 2017, pp. 181-184.
L. A. da Luz, M. S. A. de Azevedo, V. M. dos Santos, L. A. Reale, F. S. Andrade, K. R. M. Brito, and J. L. dos Santos, “Pênfigo vulgar em homem jovem: relato de caso e revisão da literatura,” Brasília Med, vol. 50, no. 4, pp. 346-353, 2013.
P. Shi, V. A. Peris, A. Echiadis, J. Zheng, Y.-S. Zhu, P. Y. S. Cheang, and S.-J. Hu, “Non-contact reflection photoplethysmography towards effective human physiological monitoring,” Journal of Medical and Biological Engineering, vol. 30, no. 3, pp. 161-167, 2009.
M. Bonvento, “Acessos vasculares e infecção relacionada à cateter,” Revista Brasileira de terapia intensiva, vol. 19, no. 2, pp. 226-230, 2007.
P. S. Addison, D. Jacquel, D. M. Foo, A. Antunes, and U. R. Borg, “Video-based physiologic monitoring during an acute hypoxic challenge: heart rate, respiratory rate, and oxygen saturation,” Anesthesia & Analgesia, vol. 125, no. 3, pp. 860-873, 2017.
D. Fokam and C. Lehmann, “Clinical assessment of arthritic knee pain by infrared thermography,” Journal of basic and clinical physiology and pharmacology, vol. 30, no. 3, 2019.
L. I. Castillo, M. E. Browne, T. Hadjistavropoulos, K. M. Prkachin, and R. Goubran, “Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia,” Journal of Rehabilitation and Assistive Technologies Engineering, vol. 7, p. 2055668320950196, 2020.
X. Chen, J. Cheng, R. Song, Y. Liu, R. Ward, and Z. J. Wang, “Videobased heart rate measurement: Recent advances and future prospects,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 10, pp. 3600-3615, 2018.
K. M. Prkachin and P. E. Solomon, “The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain,” Pain, vol. 139, no. 2, pp. 267-274, 2008.
P. Ekman, W. V. Friesen, and J. C. Hager, “Facial action coding system: The manual on cd rom,” A Human Face, Salt Lake City, 2002.
S. Hinduja, S. Canavan, and G. Kaur, “Multimodal fusion of physiological signals and facial action units for pain recognition,” in IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 2020, pp. 577-581.
P. Lucey, J. F. Cohn, K. M. Prkachin, P. E. Solomon, and I. Matthews, “Painful data: The unbc-mcmaster shoulder pain expression archive database,” in IEEE International Conference on Automatic Face & Gesture Recognition, 2011, pp. 57-64.
A. B. Hertzman and J. B. Dillon, “Applications of photoelectric plethysmography in peripheral vascular disease,” American Heart Journal, vol. 20, no. 6, pp. 750-761, 1940.
M. A. Hassan, A. S. Malik, D. Fofi, N. Saad, B. Karasfi, Y. S. Ali, and F. Meriaudeau, “Heart rate estimation using facial video: A review,” Biomedical Signal Processing and Control, vol. 38, pp. 346-360, 2017.
H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. Freeman, “Eulerian video magnification for revealing subtle changes in the world,” ACM transactions on graphics (TOG), vol. 31, no. 4, pp. 1-8, 2012.
S. L. Bennett, R. Goubran, and F. Knoefel, “Adaptive eulerian video magnification methods to extract heart rate from thermal video,” in IEEE International Symposium on medical measurements and applications. IEEE, 2016, pp. 1-5.
Y. S. Dosso, A. Bekele, and J. R. Green, “Eulerian magnification of multi-modal rgb-d video for heart rate estimation,” in International Symposium on Medical Measurements and Applications. IEEE, 2018, pp. 1-6.
E. Moya-Albor, J. Brieva, H. Ponce, and L. Martínez-Villaseñor, “A non-contact heart rate estimation method using video magnification and neural networks,” IEEE Instrumentation & Measurement Magazine, vol. 23, no. 4, pp. 56-62, 2020.
A. Alzahrani, J. Hosseinkhani, S. Rajan, and E. Ukwatta, “Reducing motion impact on video magnification using wavelet transform and principal component analysis for heart rate estimation,” in IEEE International Instrumentation and Measurement Technology Conference. IEEE, 2021, pp. 1-6.
A. Alarifi, A. Tolba, and A. S. Hassanein, “Visualization process assisted by the eulerian video magnification algorithm for a heart rate monitoring system: mobile applications,” Multimedia Tools and Applications, vol. 79, no. 7, pp. 5149-5160, 2020.
N. Boyko, O. Basystiuk, and N. Shakhovska, “Performance evaluation and comparison of software for face recognition, based on dlib and opencv library,” in IEEE Second International Conference on Data Stream Mining & Processing. IEEE, 2018, pp. 478-482.
A. M. Rodríguez and J. R. Castro, “Pulse rate variability analysis by video using face detection and tracking algorithms,” in Annual international conference of the ieee engineering in medicine and biology society. IEEE, 2015, pp. 5696-5699.
E. O. Brigham and R. Morrow, “The fast fourier transform,” IEEE spectrum, vol. 4, no. 12, pp. 63-70, 1967.
C. A. Corneanu, M. O. Simón, J. F. Cohn, and S. E. Guerrero, “Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 8, pp. 1548-1568, 2016.
B. Farnsworth, “Facial action coding system (facs)””a visual guidebook,” https://imotions.com/blog/facial-action-coding-system/, 2016.
A. Semwal and N. D. Londhe, “Computer aided pain detection and intensity estimation using compact cnn based fusion network,” Applied Soft Computing, vol. 112, p. 107780, 2021.
T. B. Fitzpatrick, “The validity and practicality of sun-reactive skin types i through vi,” Archives of dermatology, vol. 124, no. 6, pp. 869-871, 1988.
C.-Y. Chi, C.-H. Chen, C.-C. Feng, and C.-Y. Chen, “Fundamentals of statistical signal processing,” Blind Equalization and System Identification: Batch Processing Algorithms, Performance and Applications, pp. 83-182, 2006.
J. Souza, T. de Oliveira, C. Casa, and A. Ortoncelli, “Automatic detection of lupus butterfly malar rash based on transfer learning,” in Workshop de Visão Computacional. SBC, 2020, pp. 36-40.
Í. Gama, A. Coelho, and M. Baffa, “Fundus eye images classification for diabetic retinopathy detection using very deep convolutional neural network,” in Workshop de Visão Computacional. SBC, 2020, pp. 24-29.
M. Benjamin and K. CB, “Miller-keane encyclopedia and dictionary of medicine,” Nursing and Allied Health. Philadelphia: Saunders, 1997.
S. N. Raja, D. B. Carr, M. Cohen, N. B. Finnerup, H. Flor, S. Gibson, F. J. Keefe, J. S. Mogil, M. Ringkamp, K. A. Sluka et al., “The revised international association for the study of pain definition of pain: concepts, challenges, and compromises,” Pain, vol. 161, no. 9, pp. 1976- 1982, 2020.
N. Katz, “The impact of pain management on quality of life,” Journal of pain and symptom management, vol. 24, no. 1, pp. S38-S47, 2002.
A. B. Amspoker, A. L. Snow, B. N. Renn, P. Block, S. Pickens, R. O. Morgan, and M. E. Kunik, “Patient versus informal caregiver proxy reports of pain interference in persons with dementia,” Journal of Applied Gerontology, vol. 40, no. 4, pp. 414-422, 2021.
L. P. Carlini, T. M. Heideirich, R. C. Balda, M. C. Barros, R. Guinsburg, and C. E. Thomaz, “Visual perception of pain in neonatal face images,” in Workshop de Visão Computacional. SBC, 2019, pp. 37-42.
L. Buzuti, T. Heideirich, M. Barros, R. Guinsburg, and C. Thomaz, “Neonatal pain assessment from facial expression using deep neural networks,” in Workshop de Visão Computacional, 2020, pp. 87-92.
K. Herr, P. J. Coyne, E. Ely, C. Gélinas, and R. C. Manworren, “Pain assessment in the patient unable to self-report: clinical practice recommendations in support of the aspmn 2019 position statement,” Pain Management Nursing, vol. 20, no. 5, pp. 404-417, 2019.
J. O. Egede, S. Song, T. A. Olugbade, C. Wang, C. D. C. Amanda, H. Meng, M. Aung, N. D. Lane, M. Valstar, and N. Bianchi-Berthouze, “Emopain challenge 2020: Multimodal pain evaluation from facial and bodily expressions,” in IEEE International Conference on Automatic Face and Gesture Recognition, 2020, pp. 849-856.
D. Lopez-Martinez and R. Picard, “Multi-task neural networks for personalized pain recognition from physiological signals,” in International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, 2017, pp. 181-184.
L. A. da Luz, M. S. A. de Azevedo, V. M. dos Santos, L. A. Reale, F. S. Andrade, K. R. M. Brito, and J. L. dos Santos, “Pênfigo vulgar em homem jovem: relato de caso e revisão da literatura,” Brasília Med, vol. 50, no. 4, pp. 346-353, 2013.
P. Shi, V. A. Peris, A. Echiadis, J. Zheng, Y.-S. Zhu, P. Y. S. Cheang, and S.-J. Hu, “Non-contact reflection photoplethysmography towards effective human physiological monitoring,” Journal of Medical and Biological Engineering, vol. 30, no. 3, pp. 161-167, 2009.
M. Bonvento, “Acessos vasculares e infecção relacionada à cateter,” Revista Brasileira de terapia intensiva, vol. 19, no. 2, pp. 226-230, 2007.
P. S. Addison, D. Jacquel, D. M. Foo, A. Antunes, and U. R. Borg, “Video-based physiologic monitoring during an acute hypoxic challenge: heart rate, respiratory rate, and oxygen saturation,” Anesthesia & Analgesia, vol. 125, no. 3, pp. 860-873, 2017.
D. Fokam and C. Lehmann, “Clinical assessment of arthritic knee pain by infrared thermography,” Journal of basic and clinical physiology and pharmacology, vol. 30, no. 3, 2019.
L. I. Castillo, M. E. Browne, T. Hadjistavropoulos, K. M. Prkachin, and R. Goubran, “Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia,” Journal of Rehabilitation and Assistive Technologies Engineering, vol. 7, p. 2055668320950196, 2020.
X. Chen, J. Cheng, R. Song, Y. Liu, R. Ward, and Z. J. Wang, “Videobased heart rate measurement: Recent advances and future prospects,” IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 10, pp. 3600-3615, 2018.
K. M. Prkachin and P. E. Solomon, “The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain,” Pain, vol. 139, no. 2, pp. 267-274, 2008.
P. Ekman, W. V. Friesen, and J. C. Hager, “Facial action coding system: The manual on cd rom,” A Human Face, Salt Lake City, 2002.
S. Hinduja, S. Canavan, and G. Kaur, “Multimodal fusion of physiological signals and facial action units for pain recognition,” in IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 2020, pp. 577-581.
P. Lucey, J. F. Cohn, K. M. Prkachin, P. E. Solomon, and I. Matthews, “Painful data: The unbc-mcmaster shoulder pain expression archive database,” in IEEE International Conference on Automatic Face & Gesture Recognition, 2011, pp. 57-64.
A. B. Hertzman and J. B. Dillon, “Applications of photoelectric plethysmography in peripheral vascular disease,” American Heart Journal, vol. 20, no. 6, pp. 750-761, 1940.
M. A. Hassan, A. S. Malik, D. Fofi, N. Saad, B. Karasfi, Y. S. Ali, and F. Meriaudeau, “Heart rate estimation using facial video: A review,” Biomedical Signal Processing and Control, vol. 38, pp. 346-360, 2017.
H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, and W. Freeman, “Eulerian video magnification for revealing subtle changes in the world,” ACM transactions on graphics (TOG), vol. 31, no. 4, pp. 1-8, 2012.
S. L. Bennett, R. Goubran, and F. Knoefel, “Adaptive eulerian video magnification methods to extract heart rate from thermal video,” in IEEE International Symposium on medical measurements and applications. IEEE, 2016, pp. 1-5.
Y. S. Dosso, A. Bekele, and J. R. Green, “Eulerian magnification of multi-modal rgb-d video for heart rate estimation,” in International Symposium on Medical Measurements and Applications. IEEE, 2018, pp. 1-6.
E. Moya-Albor, J. Brieva, H. Ponce, and L. Martínez-Villaseñor, “A non-contact heart rate estimation method using video magnification and neural networks,” IEEE Instrumentation & Measurement Magazine, vol. 23, no. 4, pp. 56-62, 2020.
A. Alzahrani, J. Hosseinkhani, S. Rajan, and E. Ukwatta, “Reducing motion impact on video magnification using wavelet transform and principal component analysis for heart rate estimation,” in IEEE International Instrumentation and Measurement Technology Conference. IEEE, 2021, pp. 1-6.
A. Alarifi, A. Tolba, and A. S. Hassanein, “Visualization process assisted by the eulerian video magnification algorithm for a heart rate monitoring system: mobile applications,” Multimedia Tools and Applications, vol. 79, no. 7, pp. 5149-5160, 2020.
N. Boyko, O. Basystiuk, and N. Shakhovska, “Performance evaluation and comparison of software for face recognition, based on dlib and opencv library,” in IEEE Second International Conference on Data Stream Mining & Processing. IEEE, 2018, pp. 478-482.
A. M. Rodríguez and J. R. Castro, “Pulse rate variability analysis by video using face detection and tracking algorithms,” in Annual international conference of the ieee engineering in medicine and biology society. IEEE, 2015, pp. 5696-5699.
E. O. Brigham and R. Morrow, “The fast fourier transform,” IEEE spectrum, vol. 4, no. 12, pp. 63-70, 1967.
C. A. Corneanu, M. O. Simón, J. F. Cohn, and S. E. Guerrero, “Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 8, pp. 1548-1568, 2016.
B. Farnsworth, “Facial action coding system (facs)””a visual guidebook,” https://imotions.com/blog/facial-action-coding-system/, 2016.
A. Semwal and N. D. Londhe, “Computer aided pain detection and intensity estimation using compact cnn based fusion network,” Applied Soft Computing, vol. 112, p. 107780, 2021.
T. B. Fitzpatrick, “The validity and practicality of sun-reactive skin types i through vi,” Archives of dermatology, vol. 124, no. 6, pp. 869-871, 1988.
C.-Y. Chi, C.-H. Chen, C.-C. Feng, and C.-Y. Chen, “Fundamentals of statistical signal processing,” Blind Equalization and System Identification: Batch Processing Algorithms, Performance and Applications, pp. 83-182, 2006.
J. Souza, T. de Oliveira, C. Casa, and A. Ortoncelli, “Automatic detection of lupus butterfly malar rash based on transfer learning,” in Workshop de Visão Computacional. SBC, 2020, pp. 36-40.
Í. Gama, A. Coelho, and M. Baffa, “Fundus eye images classification for diabetic retinopathy detection using very deep convolutional neural network,” in Workshop de Visão Computacional. SBC, 2020, pp. 24-29.
Publicado
22/11/2021
Como Citar
SOUZA, Jhonatan; CASA, Claudemir; ORTONCELLI, André Roberto.
Analysis of a Video-Based Pain Monitoring System in Raspberry Pi. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 17. , 2021, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 195-200.
DOI: https://doi.org/10.5753/wvc.2021.18913.
