Abstract
Inertial Measurement Unit (IMU) sensors are present in everyday devices such as smartphones and fitness watches. As a result, the array of health-related research and applications that tap onto this data has been growing, but little attention has been devoted to the prediction of an individual’s heart rate (HR) from IMU data, when undergoing a physical activity. Would that be even possible? If so, this could be used to design personalized sets of aerobic exercises, for instance. In this work, we show that it is viable to obtain accurate HR predictions from IMU data using Recurrent Neural Networks, provided only access to HR and IMU data from a short-lived, previously executed activity. We propose a novel method for initializing an RNN’s hidden state vectors, using a specialized network that attempts to extract an embedding of the physical conditioning (PCE) of a subject. We show that using a discriminator in the training phase to help the model learn whether two PCEs belong to the same individual further reduces the prediction error. We evaluate the proposed model when predicting the HR of 23 subjects performing a variety of physical activities from IMU data available in public datasets (PAMAP2, PPG-DaLiA). For comparison, we use as baselines the only model specifically proposed for this task and an adapted state-of-the-art model for Human Activity Recognition (HAR), a closely related task. Our method, PCE-LSTM, yields over 10% lower mean absolute error. We demonstrate empirically that this error reduction is in part due to the use of the PCE. Last, we use the two datasets (PPG-DaLiA, WESAD) to show that PCE-LSTM can also be successfully applied when photoplethysmography (PPG) sensors are available, outperforming the state-of-the-art deep learning baselines by more than 30%.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We released all our code at https://github.com/davipeag/HeartRateRegression.
- 2.
We avoid overfitting by using disjoint sets of subjects for training and validation.
- 3.
Alternatively, losses’ weights can be set by hyperparameter tuning, but since we use a single subject for validation, we fixed the weights to (0.9, 0.1) to avoid overfitting to the validation subject.
References
Reiss, A., Indlekofer, I., Schmidt, P., Van Laerhoven, K.: Deep PPG: large-scale heart rate estimation with convolutional neural networks. Sensors 19(14), 3079 (2019)
Ludwig, M., Hoffmann, K., Endler, S., Asteroth, A., Wiemeyer, J.: Measurement, prediction, and control of individual heart rate responses to exercise-basics and options for wearable devices. Front. Physiol. 9, 778 (2018)
Cheng, T.M., Savkin, A.V., Celler, B.G., Wang, L., Su, S.W.: A nonlinear dynamic model for heart rate response to treadmill walking exercise. In: IEEE IEMBS, pp. 2988–2991 (2007)
Hunt, K.J., Fankhauser, S.E.: Heart rate control during treadmill exercise using input-sensitivity shaping for disturbance rejection of very-low-frequency heart rate variability. Biomed. Signal Process. Control 30, 31–42 (2016)
Mohammad, S., Guerra, T.M., Grobois, J.M., Hecquet, B.: Heart rate control during cycling exercise using Takagi-Sugeno models. IFAC Proc. 44(1), 12783–12788 (2011)
Zhang, H., Wen, B., Liu, J.: The prediction of heart rate during running using Bayesian combined predictor. In: IEEE IWCMC, pp. 981–986 (2018)
Xiao, F., Yuchi, M., Ding, M., Jo, J.: A research of heart rate prediction model based on evolutionary neural network. In: IEEE ICBMI, pp. 304–307 (2011)
Yuchi, M., Jo, J.: Heart rate prediction based on physical activity using feedforwad neural network. In: IEEE ICHIT, pp. 344–350 (2008)
Mohajerin, N., Waslander, S.L.: Multistep prediction of dynamic systems with recurrent neural networks. 30(11), 3370–3383 (2019)
Salehizadeh, S., Dao, D., Bolkhovsky, J., Cho, C., Mendelson, Y., Chon, K.: A novel time-varying spectral filtering algorithm for reconstruction of motion artifact corrupted heart rate signals during intense physical activities using a wearable photoplethysmogram sensor. Sensors 16(1), 10 (2016)
Schäck, T., Muma, M., Zoubir, A.M.: Computationally efficient heart rate estimation during physical exercise using photoplethysmographic signals. In: EUSIPCO, pp. 2478–2481 (2017)
Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., Van Laerhoven, K.: Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In: ACM ICMI, pp. 400–408, New York, NY, USA (2018). Association for Computing Machinery
Reiss, A., Stricker, D.: Introducing a new benchmarked dataset for activity monitoring. In: IEEE ISWC, pp. 108–109, June 2012
van Gent, P., Farah, H., van Nes, N., van Arem, B.: Analysing noisy driver physiology real-time using off-the-shelf sensors: heart rate analysis software from the taking the fast lane project. J. Open Res. Softw. 7(1) (2019)
Eyobu, O.S., Han, D.S.: Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network. Sensors 18(9), 2892 (2018)
Rueda, F.M., Grzeszick, R., Fink, G.A., Feldhorst, S., Hompel, M.T.: Convolutional neural networks for human activity recognition using body-worn sensors. Informatics 5(2), 26 (2018)
Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)
Jordao, A., Nazare Jr, A.C., Sena, J., Schwartz, W.R.: Human activity recognition based on wearable sensor data: a standardization of the state-of-the-art. arXiv e-prints, page arXiv:1806.05226, June 2018
de Aguiar, D.P., Murai, F.: Am I fit for this physical activity? Neural embedding of physical conditioning from inertial sensors (2021)
Hammerla, N.Y., Halloran, S., Plötz, T.: Deep, convolutional, and recurrent models for human activity recognition using wearables. In: IJCAI, pp. 1533–1540 (2016)
Vaswani, A.: Attention is all you need. In: NeurIPS, pp. 5998–6008 (2017)
Wu, N., Green, B., Ben, X., O’Banion, S.: Deep transformer models for time series forecasting: the influenza prevalence case. arXiv e-prints, page arXiv:2001.08317, January 2020
Wang, J., Chen, Y., Hao, S., Peng, X., Lisha, H.: Deep learning for sensor-based activity recognition: a survey. Patt. Recogn. Lett. 119, 3–11 (2019)
Zimmermann, H., Grothmann, R., Schaefer, A., Tietz, C.: 8 modeling large dynamical systems with dynamical consistent neural networks. In: Haykin, S., Principe, J.C., Sejnowski, T.J., McWhirter, J. (eds.) New Directions in Statistical Signal Processing: From Systems to Brains, chapter 8. The MIT Press (2006)
Wenke, S., Fleming, J.: Contextual recurrent neural networks (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
de Aguiar, D.P., Murai, F. (2021). Encoding Physical Conditioning from Inertial Sensors for Multi-step Heart Rate Estimation. In: Britto, A., Valdivia Delgado, K. (eds) Intelligent Systems. BRACIS 2021. Lecture Notes in Computer Science(), vol 13074. Springer, Cham. https://doi.org/10.1007/978-3-030-91699-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-91699-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91698-5
Online ISBN: 978-3-030-91699-2
eBook Packages: Computer ScienceComputer Science (R0)