What do secondary studies on chatbots in education reveal?

• Leo Natan Paschoal University of São Paulo http://orcid.org/0000-0003-1076-9174
• Tayana Uchôa Conte Federal University of Amazonas https://orcid.org/0000-0001-6436-3773
• Simone R. S. de Souza University of São Paulo http://orcid.org/0000-0001-9007-9821

DOI: https://doi.org/10.5753/sbie.2022.225686

Abstract

Chatbots are established with multiple intentions to support educational activities. The number of primary studies on the educational use of chatbots has been increasing. Consequently, secondary studies have emerged in state-ofthe-art. However, there are no tertiary studies to provide a critical overview of the available literature. In particular, no understanding of what evidence has already been obtained by secondary studies and what still needs to be developed by the community. This paper seeks to alleviate this gap by categorising the secondary studies on educational chatbots. This paper offers an inventory of systematic mappings and systematic reviews. Hoping it will be helpful to researchers and practitioners interested in the field.

Keywords: Chatbots, Education, Tertiary study

References

AbuShawar, B. and Atwell, E. (2015). Alice chatbot: Trials and outputs. Computación y Sistemas, 19(4):625–632.

Allouch, M., Azaria, A., and Azoulay, R. (2021). Conversational agents: Goals, technologies, vision and challenges. Sensors, 21(24):8448.

Ayedoun, E., Hayashi, Y., and Seta, K. (2015). A conversational agent to encourage willingness to communicate in the context of english as a foreign language. Procedia Computer Science, 60:1433–1442.

Benotti, L., Martínez, M. C., and Schapachnik, F. (2014). Engaging high school students using chatbots. In Conference on Innovation and Technology in Computer Science Education, page 63–68.

Benotti, L., Martnez, M. C., and Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2):179–192.

Bezverhny, E., Dadteev, K., Barykin, L., Nemeshaev, S., and Klimov, V. (2020). Use of chat bots in learning management systems. Procedia Computer Science, 169:652–655.

Bibauw, S., François, T., and Desmet, P. (2019). Discussing with a computer to practice a foreign language: research synthesis and conceptual framework of dialogue-based call. Computer Assisted Language Learning, 32(8):827–877.

Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., and Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80(4):571–583.

Frangoudes, F., Hadjiaros, M., Schiza, E. C., Matsangidou, M., Tsivitanidou, O., and Neokleous, K. (2021). An overview of the use of chatbots in medical and healthcare education. In Zaphiris, P. and Ioannou, A., editors, Learning and Collaboration Technologies: Games and Virtual Environments for Learning, pages 170–184, Cham. Springer International Publishing.

Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., and Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of chatbot and human task partners. Computers in Human Behavior, 75:461–468.

Han, S. and Lee, M. K. (2022). Faq chatbot and inclusive learning in massive open online courses. Computers & Education, 179:104395.

Herpich, F., Nunes, F. B., Voss, G. B., and Medina, R. D. (2016). Three-dimensional virtual environment and npc: A perspective about intelligent agents ubiquitous. In Handbook of Research on 3-D Virtual Environments and Hypermedia for Ubiquitous Learning, pages 510–536. IGI Global.

Hobert, S. (2019). How are you, chatbot? evaluating chatbots in educational settings – results of a literature review. Lecture Notes in Informatics, pages 259–270.

Jeon, J. (2022). Exploring ai chatbot affordances in the efl classroom: young learners’ experiences and perspectives. Computer Assisted Language Learning, 0(0):1–26.

Kitchenham, B. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report 2.3, Keele University and University of Durham, Keele, UK.

Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., and Linkman, S. (2009). Systematic literature reviews in software engineering – a systematic literature review. Information and Software Technology, 51(1):7–15.

Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., and Linkman, S. (2010). Systematic literature reviews in software engineering – a tertiary study. Information and Software Technology, 52(8):792–805.

Maicher, K., Danforth, D., Price, A., Zimmerman, L., Wilcox, B., Liston, B., Cronau, H., Belknap, L., Ledford, C., Way, D., et al. (2017). Developing a conversational virtual standardized patient to enable students to practice history-taking skills. Simulation in Healthcare, 12(2):124–131.

Nguyen, H. D., Pham, V. T., Tran, D. A., and Le, T. T. (2019). Intelligent tutoring chatbot for solving mathematical problems in high-school. In International Conference on Knowledge and Systems Engineering, pages 1–6.

Ong, E. C. J. and Soriano, Z. C. (2016). A conversational agent to shift students’ affect state. In Baldoni, M., Baroglio, C., Bex, F., Grasso, F., Green, N., Namazi-Rad, M.-R., Numao, M., and Suarez, M. T., editors, Principles and Practice of Multi-Agent Systems, pages 86–97, Cham. Springer International Publishing.

Paladines, J. and Ramirez, J. (2020). A systematic literature review of intelligent tutoring systems with dialogue in natural language. IEEE Access, 8:164246–164267.

Paschoal, L. N., Loureiro Krassmann, A., Nunes, F. B., Morais de Oliveira, M., Bercht, M., Barbosa, E. F., and do Rocio Senger de Souza, S. (2020). A systematic identification of pedagogical conversational agents. In IEEE Frontiers in Education Conference, pages 1–9.

Paschoal, L. N., Melo, S. M., Neves, V. d. O., Conte, T. U., and de SOUZA, S. d. R. S. (2022). An experimental study on a conversational agent in software testing lessons. Informatics in Education.

Paschoal, L. N., Turci, L. F., Conte, T. U., and Souza, S. R. S. (2019). Towards a conversational agent to support the software testing education. In Brazilian Symposium on Software Engineering, page 57–66.

Pereira, J., Fernández-Raga, M., Osuna-Acedo, S., Roura-Redondo, M., Almazán-López, O., and Buldón-Olalla, A. (2019). Promoting learners’ voice productions using chatbots as a tool for improving the learning process in a mooc. Technology, Knowledge and Learning, 24(4):545–565.

Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1 – 18.

Pérez, J. Q., Daradoumis, T., and Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6):1549–1565.

Ranoliya, B. R., Raghuwanshi, N., and Singh, S. (2017). Chatbot for university related faqs. In International Conference on Advances in Computing, Communications and Informatics, pages 1525–1530.

Santos-Perez, M., Gonzalez-Parada, E., and Cano-garcia, J. M. (2013). Mobile embodied conversational agent for task specific applications. IEEE Transactions on Consumer Electronics, 59(3):61o–614.

Wik, P. and Hjalmarsson, A. (2009). Embodied conversational agents in computer assisted language learning. Speech communication, 51(10):1024–1037.

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.