Políticas Aproximadas e Parciais Sensíveis a Risco para o Controle da Propagação de Doenças Infecciosas
Abstract
Markov Decision Processes (MDPs) can be used for controlling the spread of infectious diseases and finding an optimal vaccination control policy. However, since this is a problem involving lives, it is necessary to take into account the decision-making agent’s attitude towards the risk. Thus, in this work, we use risk-sensitive MDPs with SIR compartmental model and propose two efficient algorithms to find optimized vaccination policies that allow controlling the spread of an infectious disease, i.e. to select the number of individuals who should be vaccinated at each time period considering a parameter that represents the attitude towards the risk. The first proposed solution finds a vaccination policy that is partial and optimal w.r.t. a given risk attitude. The second proposed solution is approximated and thus can solve even larger problems. The results show that: (i) the vaccination policies depend not only on the baseline reproduction rate R0, as expected, but also on the cost and attitude towards risk of a decision-making agent; and (ii) both solutions obtain a great gain in execution time and little loss in the quality when compared with the complete and non-approximate policies.
References
Charpentier, A., Elie, R., Laurière, M., and Tran, V.-C. (2020). COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability. Mathematical Modelling of Natural Phenomena, 15:57.
Diekmann, O. and Heesterbeek, J. (2000). Mathematical Epidemiology of Infectious Diseases: model building, analysis and interpretation. John Wiley & Son.
Elie, R., Hubert, E., and Turinici, G. (2020). Contact rate epidemic control of COVID-19: an equilibrium view. Mathematical Modelling of Natural Phenomena, 15:35.
Gatto, N. M. and Schellhorn, H. (2021). Optimal control of the SIR model in the presence of transmission and treatment uncertainty. Mathematical Biosciences, 333:108539.
Hansen, E. A. and Zilberstein, S. (2001). LAO*: A heuristic search algorithm that finds solutions with loops. Artif. Intell., 129:35-62.
Libin, P. J. K., Moonens, A., Verstraeten, T., Perez-Sanjines, F., Hens, N., Lemey, P., and Nowé, A. (2021). Deep reinforcement learning for large-scale epidemic control. In ECML PKDD. Applied Data Science and Demo Track, pages 155-170.
Liu, L., Luo, X., and Chang, L. (2017). Vaccination strategies of an SIR pair approximation model with demographics on complex networks. Chaos, Solitons & Fractals, 104:282-290.
Mihatsch, O. and Neuneier, R. (2002). Risk-sensitive reinforcement learning. Machine Learning, 49(2):267-290.
Nasir, A. and Rehman, H. (2017). Optimal control for stochastic model of epidemic infections. In IBCAST, pages 278-284. IEEE.
Pastor, H. D., Borges, I. O., Freire, V., Delgado, K. V., and de Barros, L. N. (2020a). Risk-sensitive piecewise-linear policy iteration for stochastic shortest path Markov decision processes. In 19th MICAI, pages 383-395. Springer.
Pastor, H. D., Freire, V., Barros, L., and Delgado, K. V. (2020b). Políticas sensíveis ao risco para o controle da propagação de doenças infecciosas. In ENIAC, pages 366-377.
Usherwood, T., LaJoie, Z., and Srivastava, V. (2021). A model and predictions for COVID-19 considering population behavior and vaccination. Sci. Rep., 11(1):1-11.
Yaesoubi, R. and Cohen, T. (2011). Dynamic health policies for controlling the spread of emerging infections: influenza as an example. PloS one, 6(9):e24043.
Yaesoubi, R. and Cohen, T. (2016). Identifying cost-effective dynamic policies to control epidemics. Statistics in medicine, 35(28):5189-5209.
Yaesoubi, R., Havumaki, J., Chitwood, M. H., Menzies, N. A., Gonsalves, G., Salomon, J. A., Paltiel, A. D., and Cohen, T. (2021). Adaptive policies to balance health benefits and economic costs of physical distancing interventions during the COVID-19 pandemic. Medical Decision Making, 41(4):386-392.
