Usability Evaluation of CoderBot as a Pedagogical Resource in Teaching Programming
Abstract
Programming is one of the first subjects taught in software engineering courses. However, students often perceive these subjects as complex. In the face of advances in technology-mediated learning, new potential solutions have emerged, such as text-based conversational agents or chatbots. In this work, we present CoderBot, an educational pedagogical agent based on Example-Based Learning, designed to help beginner students understand programming content. We performed an exploratory study to evaluate the usability of CoderBot by using the System Usability Scale (SUS) questionnaire. The results indicate that the level of satisfaction when using CoderBot is acceptable.References
Adams, D. M., McLaren, B. M., Durkin, K., Mayer, R. E., Rittle-Johnson, B., Isotani, S., e Van Velsen, M. (2014). Using erroneous examples to improve mathematics learning with a web-based tutoring system. Computers in Human Behavior, 36:401–411.
Alves, G., Rebouças, A., e Scaico, P. (2019). Coding dojo como prática de aprendizagem colaborativa para apoiar o ensino introdutório de programação: Um estudo de caso. Em Anais do XXVII Workshop sobre Educação em Computação, páginas 276–290. SBC.
Brooke, J. et al. (1996). Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4–7.
Carreira, G., Silva, L., Mendes, A. J., e Oliveira, H. G. (2022). Pyo, a chatbot assistant for introductory programming students. Em 2022 International Symposium on Computers in Education (SIIE), páginas 1–6.
Hobert, S. (2019). Say hello to ‘coding tutor’! design and evaluation of a chatbot-based learning system supporting students to learn to program.
Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., e Daradoumis, A. (2022). Educational ai chatbots for content and language integrated learning. Applied Sciences, 12(7):3239.
McLaren, B. M., van Gog, T., Ganoe, C., Karabinos, M., e Yaron, D. (2016). The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments. Computers in Human Behavior, 55:87–99.
Ondáš, S., Pleva, M., e Hládek, D. (2019). How chatbots can be involved in the education process. Em 2019 17th International Conference on Emerging eLearning Technologies and Applications (ICETA), páginas 575–580. IEEE.
Penney, J., Pimentel, J. F., Steinmacher, I., e Gerosa, M. A. (2023). Anticipating user needs: Insights from design fiction on conversational agents for computational thinking. Em International Workshop on Chatbot Research and Design, páginas 204–219.
Pérez, J. Q., Daradoumis, T., e Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6):1549–1565.
Robins, A. V. (2019). 12 novice programmers and introductory programming. The Cambridge handbook of computing education research, página 327.
Ruan, S., Jiang, L., Xu, J., Tham, B. J.-K., Qiu, Z., Zhu, Y., Murnane, E. L., Brunskill, E., e Landay, J. A. (2019). Quizbot: A dialogue-based adaptive learning system for factual knowledge. Em Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, páginas 1–13.
Sweller, J., Van Merrienboer, J. J., e Paas, F. G. (1998). Cognitive architecture and instructional design. Educational psychology review, páginas 251–296.
Winkler, R., Hobert, S., Salovaara, A., Söllner, M., e Leimeister, J. M. (2020). Sara, the lecturer: Improving learning in online education with a scaffolding-based conversational agent. Em CHI Conference on Human Factors in Computing Systems, CHI ’20, página 1–14, New York, NY, USA. Association for Computing Machinery.
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., e Drachsler, H. (2021). Are we there yet?-a systematic literature review on chatbots in education. Frontiers in artificial intelligence, 4:654924.
Alves, G., Rebouças, A., e Scaico, P. (2019). Coding dojo como prática de aprendizagem colaborativa para apoiar o ensino introdutório de programação: Um estudo de caso. Em Anais do XXVII Workshop sobre Educação em Computação, páginas 276–290. SBC.
Brooke, J. et al. (1996). Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4–7.
Carreira, G., Silva, L., Mendes, A. J., e Oliveira, H. G. (2022). Pyo, a chatbot assistant for introductory programming students. Em 2022 International Symposium on Computers in Education (SIIE), páginas 1–6.
Hobert, S. (2019). Say hello to ‘coding tutor’! design and evaluation of a chatbot-based learning system supporting students to learn to program.
Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., e Daradoumis, A. (2022). Educational ai chatbots for content and language integrated learning. Applied Sciences, 12(7):3239.
McLaren, B. M., van Gog, T., Ganoe, C., Karabinos, M., e Yaron, D. (2016). The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments. Computers in Human Behavior, 55:87–99.
Ondáš, S., Pleva, M., e Hládek, D. (2019). How chatbots can be involved in the education process. Em 2019 17th International Conference on Emerging eLearning Technologies and Applications (ICETA), páginas 575–580. IEEE.
Penney, J., Pimentel, J. F., Steinmacher, I., e Gerosa, M. A. (2023). Anticipating user needs: Insights from design fiction on conversational agents for computational thinking. Em International Workshop on Chatbot Research and Design, páginas 204–219.
Pérez, J. Q., Daradoumis, T., e Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6):1549–1565.
Robins, A. V. (2019). 12 novice programmers and introductory programming. The Cambridge handbook of computing education research, página 327.
Ruan, S., Jiang, L., Xu, J., Tham, B. J.-K., Qiu, Z., Zhu, Y., Murnane, E. L., Brunskill, E., e Landay, J. A. (2019). Quizbot: A dialogue-based adaptive learning system for factual knowledge. Em Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, páginas 1–13.
Sweller, J., Van Merrienboer, J. J., e Paas, F. G. (1998). Cognitive architecture and instructional design. Educational psychology review, páginas 251–296.
Winkler, R., Hobert, S., Salovaara, A., Söllner, M., e Leimeister, J. M. (2020). Sara, the lecturer: Improving learning in online education with a scaffolding-based conversational agent. Em CHI Conference on Human Factors in Computing Systems, CHI ’20, página 1–14, New York, NY, USA. Association for Computing Machinery.
Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., e Drachsler, H. (2021). Are we there yet?-a systematic literature review on chatbots in education. Frontiers in artificial intelligence, 4:654924.
Published
2024-09-30
How to Cite
MIRANDA, Andre L. M.; GARCIA, Renato; ORAN, Ana Carolina; GUEDES, Gilleanes T. A.; SANTANA, Brenda S.; SILVA, Davi G.; VALLE, Pedro H. D.; SILVA, Williamson.
Usability Evaluation of CoderBot as a Pedagogical Resource in Teaching Programming. In: WORKSHOP ON BOTS IN SOFTWARE ENGINEERING (WBOTS), 1. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 11-20.
DOI: https://doi.org/10.5753/wbots.2024.3926.
