Neonatal Pain Assessment From Facial Expression Using Deep Neural Networks
Resumo
Currently, neonatal pain assessment varies among health professionals, leading to late intervention and flimsy treatment of pain in several occasions. Therefore, it is essential to understand the deficiencies of the current pattern of pain assessment tools in order to develop new ones, less subjective and susceptible to external variable influences. The aim of this paper is to investigate neonatal pain assessment using two models of Deep Learning: Neonatal Convolutional Neural Network trained end-to-end and ResNet trained using Transfer Learning. We used for training two distinct databases (COPE and Unifesp) and our results showed that the use of multi-racial databases might improve the evaluation of automatic models of neonatal pain assessment.
Palavras-chave:
Automated pain recognition, deep learning, facial expression, neonatal pain assessment, pattern recognition
Referências
G. Zamzmi, "Automatic multimodal assessment of neonatal pain," Ph.D. dissertation, University of South Florida, 2018a.
K. J. Anand and D. B. Carr, "The neuroanatomy, neurophysiology, and neurochemistry of pain, stress, and analgesia in newborns and children," Pediatric Clinics of North America, vol. 36, no. 4, pp. 795–822, 1989.
B. Golianu, E. J. Krane, K. S. Galloway, and M. Yaster, "Pediatric acute pain management," Pediatric Clinics of North America, vol. 47, no. 3, pp. 559–587, 2000.
O. F. COMMITTEE et al., "Prevention and management of procedu- ral pain in the neonate: An update." Pediatrics, vol. 137, no. 2, p. e20154271, 2016.
J. Vinall, S. P. Miller, V. Chau, S. Brummelte, A. R. Synnes, and R. E. Grunau, "Neonatal pain in relation to postnatal growth in infants born very preterm," Pain, vol. 153, no. 7, pp. 1374–1381, 2012.
M. DiLorenzo, R. Pillai Riddell, and L. Holsti, "Beyond acute pain: understanding chronic pain in infancy," Children, vol. 3, no. 4, p. 26, 2016.
S. Brummelte, R. E. Grunau, V. Chau, K. J. Poskitt, R. Brant, J. Vinall, A. Gover, A. R. Synnes, and S. P. Miller, "Procedural pain and brain development in premature newborns," Annals of neurology, vol. 71, no. 3, pp. 385–396, 2012.
R. E. Grunau, M. T. Tu, M. F. Whiteld, T. F. Oberlander, J. Weinberg, W. Yu, P. Thiessen, G. Gosse, and D. Scheifele, "Cortisol, behavior, and heart rate reactivity to immunization pain at 4 months corrected age in infants born very preterm," The Clinical journal of pain, vol. 26, no. 8, p. 698, 2010.
R. E. Grunau, J. Weinberg, and M. F. Whiteld, "Neonatal procedural pain and preterm infant cortisol response to novelty at 8 months," Pediatrics, vol. 114, no. 1, pp. e77–e84, 2004.
S. M. Walker, "Translational studies identify long-term impact of prior neonatal pain experience," Pain, vol. 158, pp. S29–S42, 2017.
A. Marchant, "‘neonates do not feel pain&: a critical review of the evidence," Bioscience Horizons: The International Journal of Student Research, vol. 7, 2014.
A. T. Bhutta and K. Anand, "Vulnerability of the developing brain: neuronal mechanisms," Clinics in perinatology, vol. 29, no. 3, pp. 357– 372, 2002.
K. Anand and F. M. Scalzo, "Can adverse neonatal experiences alter brain development and subsequent behavior?" Neonatology, vol. 77, no. 2, pp. 69–82, 2000.
R. Grunau, "Self-regulation and behavior in preterm children: effects of early pain," Progress in pain research and management, vol. 26, pp. 23–56, 2003.
B. Stevens, C. Johnston, P. Petryshen, and A. Taddio, "Premature infant pain prole: development and initial validation," The Clinical journal of pain, vol. 12, no. 1, pp. 13–22, 1996.
K. Anand, "International evidence-based group for neonatal pain con- sensus statement for the prevention and management of pain in the newborn," Arch Pediatr Adolesc Med, vol. 155, no. 2, pp. 173–180, 2001.
J. G. Zwicker, S. P. Miller, R. E. Grunau, V. Chau, R. Brant, C. Studholme, M. Liu, A. Synnes, K. J. Poskitt, M. L. Stiver et al., "Smaller cerebellar growth and poorer neurodevelopmental outcomes in very preterm infants exposed to neonatal morphine," The Journal of pediatrics, vol. 172, pp. 81–87, 2016.
R. Guinsburg, "Avaliação e tratamento da dor no recém-nascido," J Pediatr (Rio J), vol. 75, no. 3, pp. 149–60, 1999.
P. Hummel and M. van Dijk, "Pain assessment: current status and challenges," in Seminars in Fetal and Neonatal medicine, vol. 11, no. 4. Elsevier, 2006, pp. 237–245.
S. H. Simons, M. van Dijk, K. S. Anand, D. Roofthooft, R. A. van Lingen, and D. Tibboel, "Do we still hurt newborn babies?: A prospective study of procedural pain and analgesia in neonates," Archives of pediatrics & adolescent medicine, vol. 157, no. 11, pp. 1058–1064, 2003.
T. M. Heiderich, "Desenvolvimento de software para identicar a expressão facial de dor do recém-nascido." 2013.
M. Schiavenato, J. F. Byers, P. Scovanner, J. M. McMahon, Y. Xia, N. Lu, and H. He, "Neonatal pain facial expression: Evaluating the primal face of pain," Pain, vol. 138, no. 2, pp. 460–471, 2008.
M. Velana, S. Gruss, G. Layher, P. Thiam, Y. Zhang, D. Schork, V. Kessler, S. Meudt, H. Neumann, J. Kim et al., "The senseemotion database: A multimodal database for the development and systematic validation of an automatic pain-and emotion-recognition system," in IAPR Workshop on Multimodal Pattern Recognition of Social Signals in Human-Computer Interaction. Springer, 2016, pp. 127–139.[24] D. L. Martinez, O. Rudovic, D. Doughty, J. A. Subramony, and R. Pi- card, "Automatic detection of nociceptive stimuli and pain intensity from facial expressions," The Journal of Pain, vol. 18, no. 4, p. S59, 2017.
P. Rodriguez, G. Cucurull, J. Gonzàlez, J. M. Gonfaus, K. Nasrollahi, T. B. Moeslund, and F. X. Roca, "Deep pain: Exploiting long short-term memory networks for facial expression classication," IEEE transactions on cybernetics, 2017.
S. Walter, S. Gruss, H. Ehleiter, J. Tan, H. C. Traue, P. Werner, A. Al- Hamadi, S. Crawcour, A. O. Andrade, and G. M. da Silva, "The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system," in 2013 IEEE international conference on cybernetics (CYBCO). IEEE, 2013, pp. 128–131.
P. Werner, A. Al-Hamadi, and R. Niese, "Comparative learning applied to intensity rating of facial expressions of pain," International Journal of Pattern Recognition and Articial Intelligence, vol. 28, no. 05, p. 1451008, 2014.
G. Zamzmi, D. Goldgof, R. Kasturi, and Y. Sun, "Neonatal pain expression recognition using transfer learning," arXiv preprint arXiv:1807.01631, 2018b.
G. Zamzmi, R. Paul, D. Goldgof, R. Kasturi, and Y. Sun, "Pain as- sessment from facial expression: Neonatal convolutional neural network (n-cnn)," in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–7.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
F. Zhang, T. Zhang, Q. Mao, and C. Xu, "Joint pose and expression modeling for facial expression recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3359–3368.
J. Deng, J. Guo, Y. Zhou, J. Yu, I. Kotsia, and S. Zafeiriou, "Reti- naface: Single-stage dense face localisation in the wild," arXiv preprint arXiv:1905.00641, 2019.
P. Viola and M. J. Jones, "Robust real-time face detection," International journal of computer vision, vol. 57, no. 2, pp. 137–154, 2004.
R. K. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks," arXiv preprint arXiv:1505.00387, 2015.
S. Brahnam, C.-F. Chuang, F. Y. Shih, and M. R. Slack, "Machine recognition and representation of neonatal facial displays of acute pain," Articial intelligence in medicine, vol. 36, no. 3, pp. 211–222, 2006.
T. M. Heiderich, A. T. F. S. Leslie, and R. Guinsburg, "Neonatal procedural pain can be assessed by computer software that has good sensitivity and specicity to detect facial movements," Acta Paediatrica, vol. 104, no. 2, pp. e63–e69, 2015.
R. V. Grunau and K. D. Craig, "Pain expression in neonates: facial action and cry," Pain, vol. 28, no. 3, pp. 395–410, 1987.
T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural networks for machine learning, vol. 4, no. 2, pp. 26–31, 2012.
K. J. Anand and D. B. Carr, "The neuroanatomy, neurophysiology, and neurochemistry of pain, stress, and analgesia in newborns and children," Pediatric Clinics of North America, vol. 36, no. 4, pp. 795–822, 1989.
B. Golianu, E. J. Krane, K. S. Galloway, and M. Yaster, "Pediatric acute pain management," Pediatric Clinics of North America, vol. 47, no. 3, pp. 559–587, 2000.
O. F. COMMITTEE et al., "Prevention and management of procedu- ral pain in the neonate: An update." Pediatrics, vol. 137, no. 2, p. e20154271, 2016.
J. Vinall, S. P. Miller, V. Chau, S. Brummelte, A. R. Synnes, and R. E. Grunau, "Neonatal pain in relation to postnatal growth in infants born very preterm," Pain, vol. 153, no. 7, pp. 1374–1381, 2012.
M. DiLorenzo, R. Pillai Riddell, and L. Holsti, "Beyond acute pain: understanding chronic pain in infancy," Children, vol. 3, no. 4, p. 26, 2016.
S. Brummelte, R. E. Grunau, V. Chau, K. J. Poskitt, R. Brant, J. Vinall, A. Gover, A. R. Synnes, and S. P. Miller, "Procedural pain and brain development in premature newborns," Annals of neurology, vol. 71, no. 3, pp. 385–396, 2012.
R. E. Grunau, M. T. Tu, M. F. Whiteld, T. F. Oberlander, J. Weinberg, W. Yu, P. Thiessen, G. Gosse, and D. Scheifele, "Cortisol, behavior, and heart rate reactivity to immunization pain at 4 months corrected age in infants born very preterm," The Clinical journal of pain, vol. 26, no. 8, p. 698, 2010.
R. E. Grunau, J. Weinberg, and M. F. Whiteld, "Neonatal procedural pain and preterm infant cortisol response to novelty at 8 months," Pediatrics, vol. 114, no. 1, pp. e77–e84, 2004.
S. M. Walker, "Translational studies identify long-term impact of prior neonatal pain experience," Pain, vol. 158, pp. S29–S42, 2017.
A. Marchant, "‘neonates do not feel pain&: a critical review of the evidence," Bioscience Horizons: The International Journal of Student Research, vol. 7, 2014.
A. T. Bhutta and K. Anand, "Vulnerability of the developing brain: neuronal mechanisms," Clinics in perinatology, vol. 29, no. 3, pp. 357– 372, 2002.
K. Anand and F. M. Scalzo, "Can adverse neonatal experiences alter brain development and subsequent behavior?" Neonatology, vol. 77, no. 2, pp. 69–82, 2000.
R. Grunau, "Self-regulation and behavior in preterm children: effects of early pain," Progress in pain research and management, vol. 26, pp. 23–56, 2003.
B. Stevens, C. Johnston, P. Petryshen, and A. Taddio, "Premature infant pain prole: development and initial validation," The Clinical journal of pain, vol. 12, no. 1, pp. 13–22, 1996.
K. Anand, "International evidence-based group for neonatal pain con- sensus statement for the prevention and management of pain in the newborn," Arch Pediatr Adolesc Med, vol. 155, no. 2, pp. 173–180, 2001.
J. G. Zwicker, S. P. Miller, R. E. Grunau, V. Chau, R. Brant, C. Studholme, M. Liu, A. Synnes, K. J. Poskitt, M. L. Stiver et al., "Smaller cerebellar growth and poorer neurodevelopmental outcomes in very preterm infants exposed to neonatal morphine," The Journal of pediatrics, vol. 172, pp. 81–87, 2016.
R. Guinsburg, "Avaliação e tratamento da dor no recém-nascido," J Pediatr (Rio J), vol. 75, no. 3, pp. 149–60, 1999.
P. Hummel and M. van Dijk, "Pain assessment: current status and challenges," in Seminars in Fetal and Neonatal medicine, vol. 11, no. 4. Elsevier, 2006, pp. 237–245.
S. H. Simons, M. van Dijk, K. S. Anand, D. Roofthooft, R. A. van Lingen, and D. Tibboel, "Do we still hurt newborn babies?: A prospective study of procedural pain and analgesia in neonates," Archives of pediatrics & adolescent medicine, vol. 157, no. 11, pp. 1058–1064, 2003.
T. M. Heiderich, "Desenvolvimento de software para identicar a expressão facial de dor do recém-nascido." 2013.
M. Schiavenato, J. F. Byers, P. Scovanner, J. M. McMahon, Y. Xia, N. Lu, and H. He, "Neonatal pain facial expression: Evaluating the primal face of pain," Pain, vol. 138, no. 2, pp. 460–471, 2008.
M. Velana, S. Gruss, G. Layher, P. Thiam, Y. Zhang, D. Schork, V. Kessler, S. Meudt, H. Neumann, J. Kim et al., "The senseemotion database: A multimodal database for the development and systematic validation of an automatic pain-and emotion-recognition system," in IAPR Workshop on Multimodal Pattern Recognition of Social Signals in Human-Computer Interaction. Springer, 2016, pp. 127–139.[24] D. L. Martinez, O. Rudovic, D. Doughty, J. A. Subramony, and R. Pi- card, "Automatic detection of nociceptive stimuli and pain intensity from facial expressions," The Journal of Pain, vol. 18, no. 4, p. S59, 2017.
P. Rodriguez, G. Cucurull, J. Gonzàlez, J. M. Gonfaus, K. Nasrollahi, T. B. Moeslund, and F. X. Roca, "Deep pain: Exploiting long short-term memory networks for facial expression classication," IEEE transactions on cybernetics, 2017.
S. Walter, S. Gruss, H. Ehleiter, J. Tan, H. C. Traue, P. Werner, A. Al- Hamadi, S. Crawcour, A. O. Andrade, and G. M. da Silva, "The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system," in 2013 IEEE international conference on cybernetics (CYBCO). IEEE, 2013, pp. 128–131.
P. Werner, A. Al-Hamadi, and R. Niese, "Comparative learning applied to intensity rating of facial expressions of pain," International Journal of Pattern Recognition and Articial Intelligence, vol. 28, no. 05, p. 1451008, 2014.
G. Zamzmi, D. Goldgof, R. Kasturi, and Y. Sun, "Neonatal pain expression recognition using transfer learning," arXiv preprint arXiv:1807.01631, 2018b.
G. Zamzmi, R. Paul, D. Goldgof, R. Kasturi, and Y. Sun, "Pain as- sessment from facial expression: Neonatal convolutional neural network (n-cnn)," in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–7.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
F. Zhang, T. Zhang, Q. Mao, and C. Xu, "Joint pose and expression modeling for facial expression recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3359–3368.
J. Deng, J. Guo, Y. Zhou, J. Yu, I. Kotsia, and S. Zafeiriou, "Reti- naface: Single-stage dense face localisation in the wild," arXiv preprint arXiv:1905.00641, 2019.
P. Viola and M. J. Jones, "Robust real-time face detection," International journal of computer vision, vol. 57, no. 2, pp. 137–154, 2004.
R. K. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks," arXiv preprint arXiv:1505.00387, 2015.
S. Brahnam, C.-F. Chuang, F. Y. Shih, and M. R. Slack, "Machine recognition and representation of neonatal facial displays of acute pain," Articial intelligence in medicine, vol. 36, no. 3, pp. 211–222, 2006.
T. M. Heiderich, A. T. F. S. Leslie, and R. Guinsburg, "Neonatal procedural pain can be assessed by computer software that has good sensitivity and specicity to detect facial movements," Acta Paediatrica, vol. 104, no. 2, pp. e63–e69, 2015.
R. V. Grunau and K. D. Craig, "Pain expression in neonates: facial action and cry," Pain, vol. 28, no. 3, pp. 395–410, 1987.
T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural networks for machine learning, vol. 4, no. 2, pp. 26–31, 2012.
Publicado
07/10/2020
Como Citar
BUZUTI, Lucas; HEIDEIRICH, Tatiany; BARROS, Marina; GUINSBURG, Ruth; THOMAZ, Carlos.
Neonatal Pain Assessment From Facial Expression Using Deep Neural Networks. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 16. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 87-92.
DOI: https://doi.org/10.5753/wvc.2020.13486.
