Qualidade de um Ecossistema de e-Learning: Indicadores de Saúde
Resumo
Este artigo discute a importância da avaliação de qualidade em ecossistemas de software, em especial no domínio educacional. São apresentados alguns indicadores de saúde de ecossistemas, utilizando como foco o BROAD-ECOS, um Ecossistema de eLearning baseado em serviços educacionais, reuso e compartilhamento de recursos em um contexto interorganizacional. Foram definidas métricas para avaliação da saúde do ecossistema e, a partir da arquitetura semiautomatizada HEAL ME foi avaliada a qualidade do ecossistema, com quatro indicadores. Há indícios da viabilidade do processo de avaliação, e suas análises e resultados poderão ser utilizados para aperfeiçoamento do ecossistema, sua sobrevivência e adoção em larga escala.
Palavras-chave:
Saúde de ECOS, Ecossistemas de Software, E-Learning, Qualidade
Referências
G. P. Basharin, A. N. Langville, and V. A. Naumov. The life and work of a.a. Markov. Linear Algebra and its Applications, 386:3–26, 2004.
D. P. Bertsekas and H. Yu. Distributed asynchronous policy iteration in dynamic programming. In Conference on Communication, Control, and Computing, pages 1368–1375. IEEE, 2010.
V. Borkar and S. Meyn. Risk-sensitive optimal control for Markov decision processes with monotone cost. Mathematics of Operations Research, 27(1):192–209, 2002.
N. Bäuerle and U. Rieder. More risk-sensitive Markov decision processes. Mathematics of Operations Research, 39(1):105–120, 2014.
R. Cavazos-Cadena. Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space. Mathematical Methods of Operations Research, 57(2):263–285, 2003.
R. Cavazos-Cadena and E. Fernández-Gaucherand. Markov decision processes with risk-sensitive criteria: Dynamic programming operators and discounted stochastic games. In Proceedings of the IEEE Conference on Decision and Control, volume 3, pages 2110–2112, 2001.
R. Cavazos-Cadena and D. Hernández-Hernández. Discounted approximations for risk-sensitive average criteria in: Markov decision chains with finite state space. Mathematics of Operations Research, 36(1):133–146, 2011.
R. Cavazos-Cadena and R. Montes-de Oca. Nearly optimal policies in risk-sensitive positive dynamic programming on discrete spaces. Mathematical Methods of Operations Research, 52(1):133–167, 2000.
R. Cavazos-Cadena and R. Montes-De-Oca. The value iteration algorithm in risk-sensitive average Markov decision chains with finite state space. Mathematics of Operations Research, 28(4):752–776, 2003.
R. Cavazos-Cadena and F. Salem-Silva. The discounted method and equivalence of average criteria for risksensitive Markov decision processes on Borel spaces. Applied Mathematics and Optimization, 61(2):167–190, 2010.
E. Delage and S. Mannor. Percentile optimization for Markov decision processes with parameter uncertainty. Operations research, 58(1):203–213, 2010.
E. Denardo and U. Rothblum. A turnpike theorem for a risk-sensitive Markov decision process with stopping. SIAM Journal on Control and Optimization, 45(2):414– 421, 2006.
K. Dvijotham and E. Todorov. A unifying framework for linearly solvable control. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, pages 179–186, 2011.
J. Garcıa and F. Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1):1437–1480, 2015.
P. Geibel and F. Wysotzki. Risk-sensitive reinforcement learning applied to control under constraints. J. Artif. Intell. Res.(JAIR), 24:81–108, 2005.
M. Heger. Risk and reinforcement learning: Concepts and dynamic programming. Technical report, Universitat Bremen, Germany, 1994.
R. A. Howard and J. E. Matheson. Risk-sensitive markov decision processes. Management science, 18(7):356– 369, 1972.
Y. Kadota, M. Kurano, and M. Yasuda. Discounted markov decision processes with utility constraints. Computers & Mathematics with Applications, 51(2):279 – 284, 2006.
H. Kashima. Risk-sensitive learning via minimization of empirical conditional value-at-risk. IEICE TRANSACTIONS on Information and Systems, 90(12):2043–2052, 2007.
B. A. Kitchenham. Systematic review in software engineering: Where we are and where we should be going. In Proceedings of the 2Nd International Workshop on Evidential Assessment of Software Technologies, pages 1–2. ACM, 2012.
A. Kumar, V. Kavitha, and N. Hemachandra. Finite horizon risk sensitive MDP and linear programming. In Proceedings of the IEEE Conference on Decision and Control, volume 2016-February, pages 7826–7831, 2016.
Y. Liu and S. Koenig. Existence and finiteness conditions for risk-sensitive planning: First results. In 19th National Conference on Artificial Intelligence. AAAI Workshop, volume WS-04-08, pages 49–54, 2004.
Y. Liu and S. Koenig. Existence and finiteness conditions for risk-sensitive planning: Results and conjectures. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, UAI 2005, pages 354–363, 2005.
D. G. Luenberger et al. Investment science. OUP Catalogue, 1997.
H. Mausser and D. Rosen. Beyond VaR: From measuring risk to managing risk. In Proceedings of the IEEE/IAFE 1999 Conference on Computational Intelligence for Financial Engineering (CIFEr), pages 163– 178. IEEE, 1999.
R. Minami and V. F. da Silva. Shortest stochastic path with risk sensitive evaluation. In 11th Mexican International Conference on Artificial Intelligence, MICAI, pages 371–382. Springer Berlin Heidelberg, 2013.
T. M. Moldovan and P. Abbeel. Risk aversion in Markov decision processes via near optimal Chernoff bounds. In Advances in Neural Information Processing Systems, NIPS 2012, pages 3131–3139, 2012.
T. M. Moldovan and P. Abbeel. Safe exploration in Markov decision processes. In Proceedings of the 29th International Conference on Machine Learning, ICML, 2012.
G. E. Monahan. State of the art—a survey of partially observable markov decision processes: theory, models, and algorithms. Management Science, 28(1):1–16, 1982.
T. Morimura, M. Sugiyama, H. Kashima, H. Hachiya, and T. Tanaka. Nonparametric return distribution approximation for reinforcement learning. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 799–806, 2010.
T. Morimura, M. Sugiyama, H. Kashima, H. Hachiya, and T. Tanaka. Parametric return density estimation for reinforcement learning. arXiv preprint arXiv:1203.3497, 2012.
A. Nilim and L. El Ghaoui. Robust control of Markov decision processes with uncertain transition matrices. Oper. Res., 53(5):780–798, Sept. 2005.
T. Osogami. Robustness and risk-sensitivity in Markov decision processes. In Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS’12, pages 233–241, 2012.
S. Patek. On terminating Markov decision processes with a risk-averse objective function. Automatica, 37(9):1379–1386, 2001.
M. L. Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
W. A. S. Reis, K. V. Delgado, and L. N. de Barros. Distributed and asynchronous policy iteration for bounded parameter Markov decision processes. XIII Encontro Nacional de Inteligência Artificial e Computacional - ENIAC, 2016.
M. Sato. TD algorithm for the variance of return and mean-variance reinforcement learning. J. Japanese Society of Artificial Intelligence, 16(3):353–362, 2002.
Q. Wei. Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion. Math. Methods of Operations Research, 84(3):461–487, 2016.
Q. Wei and X. Chen. Continuous-time Markov decision processes under the risk-sensitive average cost criterion. Operations Research Letters, 44(4):457–462, 2016.
D. P. Bertsekas and H. Yu. Distributed asynchronous policy iteration in dynamic programming. In Conference on Communication, Control, and Computing, pages 1368–1375. IEEE, 2010.
V. Borkar and S. Meyn. Risk-sensitive optimal control for Markov decision processes with monotone cost. Mathematics of Operations Research, 27(1):192–209, 2002.
N. Bäuerle and U. Rieder. More risk-sensitive Markov decision processes. Mathematics of Operations Research, 39(1):105–120, 2014.
R. Cavazos-Cadena. Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space. Mathematical Methods of Operations Research, 57(2):263–285, 2003.
R. Cavazos-Cadena and E. Fernández-Gaucherand. Markov decision processes with risk-sensitive criteria: Dynamic programming operators and discounted stochastic games. In Proceedings of the IEEE Conference on Decision and Control, volume 3, pages 2110–2112, 2001.
R. Cavazos-Cadena and D. Hernández-Hernández. Discounted approximations for risk-sensitive average criteria in: Markov decision chains with finite state space. Mathematics of Operations Research, 36(1):133–146, 2011.
R. Cavazos-Cadena and R. Montes-de Oca. Nearly optimal policies in risk-sensitive positive dynamic programming on discrete spaces. Mathematical Methods of Operations Research, 52(1):133–167, 2000.
R. Cavazos-Cadena and R. Montes-De-Oca. The value iteration algorithm in risk-sensitive average Markov decision chains with finite state space. Mathematics of Operations Research, 28(4):752–776, 2003.
R. Cavazos-Cadena and F. Salem-Silva. The discounted method and equivalence of average criteria for risksensitive Markov decision processes on Borel spaces. Applied Mathematics and Optimization, 61(2):167–190, 2010.
E. Delage and S. Mannor. Percentile optimization for Markov decision processes with parameter uncertainty. Operations research, 58(1):203–213, 2010.
E. Denardo and U. Rothblum. A turnpike theorem for a risk-sensitive Markov decision process with stopping. SIAM Journal on Control and Optimization, 45(2):414– 421, 2006.
K. Dvijotham and E. Todorov. A unifying framework for linearly solvable control. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, pages 179–186, 2011.
J. Garcıa and F. Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1):1437–1480, 2015.
P. Geibel and F. Wysotzki. Risk-sensitive reinforcement learning applied to control under constraints. J. Artif. Intell. Res.(JAIR), 24:81–108, 2005.
M. Heger. Risk and reinforcement learning: Concepts and dynamic programming. Technical report, Universitat Bremen, Germany, 1994.
R. A. Howard and J. E. Matheson. Risk-sensitive markov decision processes. Management science, 18(7):356– 369, 1972.
Y. Kadota, M. Kurano, and M. Yasuda. Discounted markov decision processes with utility constraints. Computers & Mathematics with Applications, 51(2):279 – 284, 2006.
H. Kashima. Risk-sensitive learning via minimization of empirical conditional value-at-risk. IEICE TRANSACTIONS on Information and Systems, 90(12):2043–2052, 2007.
B. A. Kitchenham. Systematic review in software engineering: Where we are and where we should be going. In Proceedings of the 2Nd International Workshop on Evidential Assessment of Software Technologies, pages 1–2. ACM, 2012.
A. Kumar, V. Kavitha, and N. Hemachandra. Finite horizon risk sensitive MDP and linear programming. In Proceedings of the IEEE Conference on Decision and Control, volume 2016-February, pages 7826–7831, 2016.
Y. Liu and S. Koenig. Existence and finiteness conditions for risk-sensitive planning: First results. In 19th National Conference on Artificial Intelligence. AAAI Workshop, volume WS-04-08, pages 49–54, 2004.
Y. Liu and S. Koenig. Existence and finiteness conditions for risk-sensitive planning: Results and conjectures. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, UAI 2005, pages 354–363, 2005.
D. G. Luenberger et al. Investment science. OUP Catalogue, 1997.
H. Mausser and D. Rosen. Beyond VaR: From measuring risk to managing risk. In Proceedings of the IEEE/IAFE 1999 Conference on Computational Intelligence for Financial Engineering (CIFEr), pages 163– 178. IEEE, 1999.
R. Minami and V. F. da Silva. Shortest stochastic path with risk sensitive evaluation. In 11th Mexican International Conference on Artificial Intelligence, MICAI, pages 371–382. Springer Berlin Heidelberg, 2013.
T. M. Moldovan and P. Abbeel. Risk aversion in Markov decision processes via near optimal Chernoff bounds. In Advances in Neural Information Processing Systems, NIPS 2012, pages 3131–3139, 2012.
T. M. Moldovan and P. Abbeel. Safe exploration in Markov decision processes. In Proceedings of the 29th International Conference on Machine Learning, ICML, 2012.
G. E. Monahan. State of the art—a survey of partially observable markov decision processes: theory, models, and algorithms. Management Science, 28(1):1–16, 1982.
T. Morimura, M. Sugiyama, H. Kashima, H. Hachiya, and T. Tanaka. Nonparametric return distribution approximation for reinforcement learning. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 799–806, 2010.
T. Morimura, M. Sugiyama, H. Kashima, H. Hachiya, and T. Tanaka. Parametric return density estimation for reinforcement learning. arXiv preprint arXiv:1203.3497, 2012.
A. Nilim and L. El Ghaoui. Robust control of Markov decision processes with uncertain transition matrices. Oper. Res., 53(5):780–798, Sept. 2005.
T. Osogami. Robustness and risk-sensitivity in Markov decision processes. In Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS’12, pages 233–241, 2012.
S. Patek. On terminating Markov decision processes with a risk-averse objective function. Automatica, 37(9):1379–1386, 2001.
M. L. Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
W. A. S. Reis, K. V. Delgado, and L. N. de Barros. Distributed and asynchronous policy iteration for bounded parameter Markov decision processes. XIII Encontro Nacional de Inteligência Artificial e Computacional - ENIAC, 2016.
M. Sato. TD algorithm for the variance of return and mean-variance reinforcement learning. J. Japanese Society of Artificial Intelligence, 16(3):353–362, 2002.
Q. Wei. Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion. Math. Methods of Operations Research, 84(3):461–487, 2016.
Q. Wei and X. Chen. Continuous-time Markov decision processes under the risk-sensitive average cost criterion. Operations Research Letters, 44(4):457–462, 2016.
Publicado
17/05/2017
Como Citar
CARVALHO, Iuri; VEIGA, Welington; CAMPOS, Fernanda; BRAGA, Regina; STRÖELE, Victor.
Qualidade de um Ecossistema de e-Learning: Indicadores de Saúde. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 13. , 2017, Lavras.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2017
.
p. 230-237.
DOI: https://doi.org/10.5753/sbsi.2017.6047.
