Applying remote health monitoring to understand users’ QoE in multisensory applications in real-time
Abstract
User’s Quality of Experience (QoE) understanding from objective metrics has been increasingly explored in multisensory research. However, capturing physiological data adds a degree of difculty to an already complex environment composed of software to reproduce content and actuators to deliver sensory effects. In this paper, we introduce the potential use of remote patient monitoring (RPM) systems to monitor users’ QoE through a specifc tool named HealthDash. We aim to raise discussion around them in digital multisensory experiences, their application, advantages and disadvantages, and challenges and opportunities.
Keywords:
IoT, multisensory, user monitoring, health sensors
References
J. R. Celestrini, A. M. Baldi, R. V. Andreão, J. G. Pereira-Filho, and C. A. S. Santos. 2021. Flow-Based Situation-Aware Approach for eHealth Data Processing. In 2020 IEEE International Conference on E-health Networking, Application Services (HEALTHCOM). 1–7. https://doi.org/10.1109/HEALTHCOM49281.2021.9398992
A. Covaci, E. B. Saleme, G. Mesfn, N. Hussain, E. Kani-Zabihi, and G. Ghinea. 2020. How Do We Experience Crossmodal Correspondent Mulsemedia Content? IEEE Transactions on Multimedia 22, 5 (2020), 1249–1258. https: //doi.org/10.1109/TMM.2019.2941274
D. Egan, S. Brennan, J. Barrett, Y. Qiao, C. Timmerer, and N. Murray. 2016. An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). 1–6. https://doi.org/10.1109/QoMEX. 2016.7498964
F. A. C. Farias, M. Dagostini, C, Y. A. Bicca, V. F. Falavigna, and A. Falavigna. 2020. Remote Patient Monitoring: A Systematic Review. Telemedicine and e-Health 26, 5 (2020), 576–583. https://doi.org/10.1089/tmj.2019.0066 PMID: 31314689.
M. Fazio, A. Celesti, F. G. Márquez, A. Glikson, and M. Villari. 2015. Exploiting the FIWARE cloud platform to develop a remote patient monitoring system. In 2015 IEEE Symposium on Computers and Communication (ISCC). 264–270. https://doi.org/10.1109/ISCC.2015.7405526
M. Hamim, S. Paul, S. I. Hoque, M. N. Rahman, and I. Baqee. 2019. IoT Based Remote Health Monitoring System for Patients and Elderly People. In 2019 International Conference on Robotics,Electrical and Signal Processing Techniques (ICREST). 533–538. https://doi.org/10.1109/ICREST.2019.8644514
C. Keighrey, R. Flynn, S. Murray, S. Brennan, and N. Murray. 2017. Comparing User QoE via Physiological and Interaction Measurements of Immersive AR and VR Speech and Language Therapy Applications. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017 (Mountain View, California, USA) (Thematic Workshops ’17). Association for Computing Machinery, New York, NY, USA, 485–492. https://doi.org/10.1145/3126686.3126747
G. Mesfn, E. B. Saleme, O. A. Ademoye, E. Kani-Zabihi, C. A. S. Santos, and G. Ghinea. 2021. Less is (Just as Good as) More - an Investigation of Odor Intensity and Hedonic Valence in Mulsemedia QoE using Heart Rate and Eye Tracking. IEEE Transactions on Multimedia 23 (2021), 1095–1105. https://doi.org/10.1109/TMM.2020.2992948
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. A mulsemedia framework for delivering sensory effects to heterogeneous systems. Multimedia Systems 25, 4 (01 Aug 2019), 421–447. https://doi.org/10.1007/s00530-019-00618-8
D. P. Salgado, T. B. Rodrigues, C. Keighrey, R. Flynn, E. L. M. Naves, N. Murray, and F. R. Martins. 2018. A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system. Proceedings of the 9th ACM Multimedia Systems Conference, MMSys 2018 18, 517–520. https://doi.org/10.1145/3204949.3208118
C. Su, J. Hajiyev, C. J. Fu, K. C. Kao, C. H. Chang, and C. T. Chang. 2019. A novel framework for a remote patient monitoring (RPM) system with abnormality detection. Health Policy and Technology 8, 2 (2019), 157–170. https: //doi.org/10.1016/j.hlpt.2019.05.008
A. Covaci, E. B. Saleme, G. Mesfn, N. Hussain, E. Kani-Zabihi, and G. Ghinea. 2020. How Do We Experience Crossmodal Correspondent Mulsemedia Content? IEEE Transactions on Multimedia 22, 5 (2020), 1249–1258. https: //doi.org/10.1109/TMM.2019.2941274
D. Egan, S. Brennan, J. Barrett, Y. Qiao, C. Timmerer, and N. Murray. 2016. An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). 1–6. https://doi.org/10.1109/QoMEX. 2016.7498964
F. A. C. Farias, M. Dagostini, C, Y. A. Bicca, V. F. Falavigna, and A. Falavigna. 2020. Remote Patient Monitoring: A Systematic Review. Telemedicine and e-Health 26, 5 (2020), 576–583. https://doi.org/10.1089/tmj.2019.0066 PMID: 31314689.
M. Fazio, A. Celesti, F. G. Márquez, A. Glikson, and M. Villari. 2015. Exploiting the FIWARE cloud platform to develop a remote patient monitoring system. In 2015 IEEE Symposium on Computers and Communication (ISCC). 264–270. https://doi.org/10.1109/ISCC.2015.7405526
M. Hamim, S. Paul, S. I. Hoque, M. N. Rahman, and I. Baqee. 2019. IoT Based Remote Health Monitoring System for Patients and Elderly People. In 2019 International Conference on Robotics,Electrical and Signal Processing Techniques (ICREST). 533–538. https://doi.org/10.1109/ICREST.2019.8644514
C. Keighrey, R. Flynn, S. Murray, S. Brennan, and N. Murray. 2017. Comparing User QoE via Physiological and Interaction Measurements of Immersive AR and VR Speech and Language Therapy Applications. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017 (Mountain View, California, USA) (Thematic Workshops ’17). Association for Computing Machinery, New York, NY, USA, 485–492. https://doi.org/10.1145/3126686.3126747
G. Mesfn, E. B. Saleme, O. A. Ademoye, E. Kani-Zabihi, C. A. S. Santos, and G. Ghinea. 2021. Less is (Just as Good as) More - an Investigation of Odor Intensity and Hedonic Valence in Mulsemedia QoE using Heart Rate and Eye Tracking. IEEE Transactions on Multimedia 23 (2021), 1095–1105. https://doi.org/10.1109/TMM.2020.2992948
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2019. A mulsemedia framework for delivering sensory effects to heterogeneous systems. Multimedia Systems 25, 4 (01 Aug 2019), 421–447. https://doi.org/10.1007/s00530-019-00618-8
D. P. Salgado, T. B. Rodrigues, C. Keighrey, R. Flynn, E. L. M. Naves, N. Murray, and F. R. Martins. 2018. A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system. Proceedings of the 9th ACM Multimedia Systems Conference, MMSys 2018 18, 517–520. https://doi.org/10.1145/3204949.3208118
C. Su, J. Hajiyev, C. J. Fu, K. C. Kao, C. H. Chang, and C. T. Chang. 2019. A novel framework for a remote patient monitoring (RPM) system with abnormality detection. Health Policy and Technology 8, 2 (2019), 157–170. https: //doi.org/10.1016/j.hlpt.2019.05.008
Published
2021-06-21
How to Cite
CELESTRINI, Jordano R.; SALEME, Estêvão B.; MURRAY, Niall; ANDREÃO, Rodrigo V.; SANTOS, Celso A. S..
Applying remote health monitoring to understand users’ QoE in multisensory applications in real-time . In: WORKSHOP ON MULTISENSORY EXPERIENCES (SENSORYX), 1. , 2021, New York.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
DOI: https://doi.org/10.5753/sensoryx.2021.15685.
