Risk-sensitive Markov decision processes with exponential utility function: A systematic review of the literature

• Elthon Freitas University of Sao Paulo
• Karina Delgado University of Sao Paulo
• Valdinei Silva University of Sao Paulo

DOI: https://doi.org/10.5753/sbsi.2017.6045

Abstract

Markov Decision Process (MDP) has been used very eciently to solve sequential decision-making problems. There are problems in which dealing with the risks of the environment to obtain a reliable result is more important than maximizing the expected average return. MDPs that deal with this type of problem are called risk-sensitive Markov decision processes (RSMDP). This systematic review of the literature aims to identify the theoretical results and proposed algorithms to solve RSMDP problems that have an exponential utility function, evaluating their main characteristics, similarities, particularities and differences in order to allow the reader the knowledge of this tool of decision making for risk sensitive problems.

Keywords: Processos de decisão de Markov, sensível a risco, averso a risco, planejamento probabilístico, utilidade exponencial

References

R. Abraham, M. Erwig, S. Kollmansberger, and E. Seifert. Visual specifications of correct spreadsheets. In Symposium on Visual Languages and Human-Centric Computing, 2005.

V. R. Basili, G. Caldiera, and H. D. Rombach. The goal question metric approach. In Encyclopedia of Software Engineering. Wiley, 1994.

M. Burnett, C. Cook, O. Pendse, G. Rothermel, J. Summet, and C. Wallace. End-user software engineering with assertions in the spreadsheet paradigm. In 25th International Conference on Software Engineering, 2003.

J. P. Caulkins, E. L. Morrison, and T. Weidemann. Do Spreadsheet Errors Lead to Bad Decisions?, pages 44–62. IGI Global, 2008.

J. P. Caulkins, E. L. Morrison, and T. Weidemann. Spreadsheet errors and decision making: Evidence from field interviews. Journal of Organizational and End User Computing, 19(3):1 – 23, 2017.

D. M. Groenewegen and E. Visser. Integration of data validation and user interface concerns in a dsl for web applications. Software & Systems Modeling, 12(1):35–52, 2013.

M. Howard, D. LeBlanc, and J. Viega. 24 Deadly Sins of Software Security: Programming Flaws and How to Fix Them. McGraw-Hill, New York, 1 edition, 2010.

D. Jannach, T. Schmitz, B. Hofer, and F. Wotawa. Avoiding, finding and fixing spreadsheet errors – a survey of automated approaches for spreadsheet {QA}. Journal of Systems and Software, 94:129 – 150, 2014.

G. Karsai, H. Krahn, C. Pinkernell, B. Rumpe, M. Schindler, and S. Vo`I´ Llkel. Design guidelines for domain specific languages. In 9th Workshop on Domain-Specific Modeling, 2009.

J. Lawrance, R. Abraham, M. Burnett, and M. Erwig. Sharing reasoning about faults in spreadsheets: An empirical study. In Visual Languages and Human-Centric Computing, 2006.

B. R. Lawson, K. R. Baker, S. G. Powell, and L. Foster-Johnson. A comparison of spreadsheet users with di↵erent levels of experience. Omega, 37(3):579 – 590, 2009.

A. M. Lund. Measuring usability with the use questionnaire. In STC Usability SIG Newsletter. 2001.

R. R. Panko. Spreadsheet errors: What we know. what we think we can do. In Symp. of the European Spreadsheet Risks Interest Group, 2008.

R. R. Panko. Two Experiments in Reducing Overconfidence in Spreadsheet Development, pages 131–149. IGI Global, 2008.

R. R. Panko and D. N. Port. End user computing: The dark matter (and dark energy) of corporate it. In 45th Hawaii Inter. Conf. on System Sciences, 2012.

S. G. Powell, K. R. Baker, and B. Lawson. A critical review of the literature on spreadsheet errors. Decision Support Systems, 46(1):128 – 138, 2008.

C. Scadi, B. Myers, and M. Shaw. Topes: Reusable abstractions for validating data. In 30th International Conference on Software Engineering, 2008.

C. Scadi, B. Myers, and M. Shaw. Fast, Accurate Creation of Data Validation Formats by End-User Developers. 2009.