Development of a set of quality metrics for evaluating augmented reality learning experiences

• Julio Augusto da Rosa Carraro Federal University of Santa Catarina https://orcid.org/0000-0002-9517-0963
• Fabiola Augusta Dahlke Spredemann Federal University of Santa Catarina
• Eliane Pozzebon Federal University of Santa Catarina
• Josete Mazon Federal University of Santa Catarina
• Iane Franceschet de Sousa Federal University of Santa Catarina
• Alexandre Marino Costa Federal University of Santa Catarina

DOI: https://doi.org/10.5753/cbie.sbie.2020.421

Abstract

This article aims to present the development of a set of metrics for the quality evaluation of augmented reality applications related to education. Using the Goal Question Metric method, 8 classes of metrics were developed and subsequently applied to an augmented reality development project. The considered metrics were effective in mapping problems in the development process, facilitating the correction and adaptation procedures. Consequently, the use of appropriate metrics enabled a greater control over the quality of applications, generating compatible experiences with their objectives.

Keywords: Augmented Reality, Quality Metrics, Software

References

Azuma, R. T. (1997). Uma pesquisa sobre realidade aumentada. Presença: Teleoperadores e Ambientes Virtuais, 6(4):355–385.


Becker, S. A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V., and Pomerantz, J. (2018). Nmc horizon report: 2018 higher education edition. Louisville, CO:


EDUCAUSE.


Bowman, D. A., Johnson, D. B., and Hodges, L. F. (2001). Testbed evaluation of virtual environment interaction techniques. Presence: Teleoperators & Virtual Environments, 10(1):75–95.


Caldiera, Victor R Basili Gianluigi e Rombach, H. D. (1994). The goal question metric approach. Encyclopedia of software engineering, pages 528–532.


Chagas, Jonas M e da Silva, J. P. S. (2019). Um plano goal/question/metrics para aprendizado baseado em problemas na engenharia de software. In Anais da III Escola Regional de Engenharia de Software, pages 73–80. SBC.


Dexter, S. (2002). etips-educational technology integration and implementation principles. In Designing instruction for technology-enhanced learning, pages 56–70. IGI


Global.


Dey, A., Billinghurst, M., Lindeman, R. W., and Swan, J. (2018). A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI, 5:37.


Dunser, A., Grasset, R., and Billinghurst, M. (2008). ¨A survey of evaluation techniques used in augmented reality studies. Human Interface Technology Laboratory New Zealand.


Gomes, M. M. P. (2011). Metricas e medição no processo de desenvolvimento de software: estudo de caso. PhD thesis, Instituto Universitario de Lisboa.


Lopes, L. M. D., Vidotto, K. N. S., Pozzebon, E., and Ferenhof, H. A. (2019). Inovações educacionais com o uso da realidade aumentada: Uma revisao sistemática. Educação em Revista, 35.


Madureira, J. S. et al. (2017). Avaliação de equipes de desenvolvimento de software por meio de metricas orientadas a objeto. Master’s thesis, Universidade Federal de Sergipe.


Nielsen, J. (1994). Usability engineering. Morgan Kaufmann.


Pressman, Roger e Maxim, B. (2016). Engenharia de Software - 8ª Edição. McGraw Hill Brasil.


Samini, Ali e Palmerius, K. L. (2017). Popular performance metrics for evaluation of interaction in virtual and augmented reality. In 2017 International Conference on Cyberworlds (CW), pages 206–209. IEEE.


Santos, M. E. C., Chen, A., Taketomi, T., Yamamoto, G., Miyazaki, J., and Kato, H. (2013). Augmented reality learning experiences: Survey of prototype design and evaluation. IEEE Transactions on learning technologies, 7(1):38–56.


Schulmeyer, Gordon Gordon e McManus, J. I. (1992). Handbook of software quality assurance. Van Nostrand Reinhold Co.


Theden, Philipp e Colsman, H. (2013). Qualitatstechniken: Werkzeuge zur Problemlosung und standigen Verbesserung. Carl Hanser Verlag GmbH Co KG.