The use of student knowledge estimates in computer programming in sensor-free detection models of confusion emotion
Abstract
Detecting student confusion allows the computer-based learning environment to perform actions that help student regulate their confusion and benefit from it. The article presents evidence on the effects of using data on student knowledge estimates and student interaction with the environment on the performance of sensor-free models of student confusion detection in computer programming tasks. Machine learning models were trained with samples of data collected from 62 students during five months in programming classes. The results presented positive evidence supporting the study approach. The article also describes scenarios where the approach is most advantageous.
Keywords:
Computer programming, Confusion, Emotion detection, Sensor-free model
References
Arguel, A., Lockyer, L., Kennedy, G., Lodge, J. M., and Pachman, M. (2019). Seeking optimal confusion: a review on epistemic emotion management in interactive digital learning environments. Interactive Learning Environments, 27:200–210.
Arroyo, I., Cooper, D., Burleson, W., Woolf, B., Muldner, K., and Christopherson, R. (2009). Emotion sensors go to school. In Frontiers in Artificial Intelligence and Applications, number 1 in Frontiers in Artificial Intelligence and Applications, pages 17–24. IOS Press.
Badrinath, A., Wang, F., and Pardos, Z. (2021). pybkt: An accessible python library of bayesian knowledge tracing models. International Educational Data Mining Society.
Bosch, N., Chen, Y., and D’Mello, S. (2014). It’s written on your face: Detecting affective states from facial expressions while learning computer programming. In TrausanMatu, S., Boyer, K. E., Crosby, M., and Panourgia, K., editors, Intelligent Tutoring Systems, pages 39–44, Cham. Springer International Publishing.
Bosch, N. and D’Mello, S. (2017). The affective experience of novice computer programmers. International Journal of Artificial Intelligence in Education, 27(1):181–206.
Bosch, N., D’Mello, S., and Mills, C. (2013). What emotions do novices experience during their first computer programming learning session? In International Conference on Artificial Intelligence in Education, pages 11–20. Springer.
Botelho, A. F., Baker, R. S., and Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In André, E., Baker, R., Hu, X., Rodrigo, M. M. T., and du Boulay, B., editors, Artificial Intelligence in Education, pages 40–51, Cham. Springer International Publishing.
Breiman, L. (2001). Random forests. Machine Learning, 45:5–32.
Chen, T. and Guestrin, C. (2016). Xgboost. volume 42, pages 785–794. ACM.
Chi, M. T. and Ohlsson, S. (2005). Complex Declarative Learning. Cambridge University Press.
Coto, M., Mora, S., Grass, B., and Murillo-Morera, J. (2021). Emotions and programming learning: systematic mapping. Computer Science Education, pages 1–36.
D’Mello, S., Person, N., and Lehman, B. (2009). Antecedent-consequent relationships and cyclical patterns between affective states and problem solving outcomes. Frontiers in Artificial Intelligence and Applications, 200:57–64.
D’Mello, S. K. (2020). Big data in the science of learning. In Big data in psychological research., pages 203–225. American Psychological Association.
D’Mello, S., Lehman, B., Pekrun, R., and Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29:153 – 170.
D’Mello, S. K. and Graesser, A. C. (2014). Confusion. In International handbook of emotions in education, pages 299–320. Routledge.
Felipe, D. A. M., Gutierrez, K. I. N., Quiros, E. C. M., and Vea, L. A. (2012). Towards the development of intelligent agent for novice c/c++ programmers through affective analysis of event logs. In Proc. Int. MultiConference Eng. Comput. Sci, volume 1, page 2012. Citeseer.
Graesser, A., Chipman, P., King, B., McDaniel, B., and D’Mello, S. (2007). Emotions and learning with autotutor. In Proceedings of the 2007 Conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, page 569–571, NLD. IOS Press.
Grafsgaard, J. F., Boyer, K. E., and Lester, J. C. (2011). Predicting facial indicators of confusion with hidden markov models. In D’Mello, S., Graesser, A., Schuller, B., and Martin, J.-C., editors, Affective Computing and Intelligent Interaction, pages 97–106, Berlin, Heidelberg. Springer Berlin Heidelberg.
Kubat, M. (2017). An introduction to machine learning, volume 2. Springer.
Kölling, M., Quig, B., Patterson, A., and Rosenberg, J. (2003). The bluej system and its pedagogy. Computer Science Education, 13:249–268.
Lee, D. M. C., Rodrigo, M. M. T., d Baker, R. S., Sugay, J. O., and Coronel, A. (2011). Exploring the relationship between novice programmer confusion and achievement. In International Conference on Affective Computing and Intelligent Interaction, pages 175–184. Springer.
Lehman, B., D’Mello, S., Strain, A., Mills, C., Gross, M., Dobbins, A., Wallace, P., Millis, K., and Graesser, A. (2013). Inducing and tracking confusion with contradictions during complex learning. International Journal of Artificial Intelligence in Education, 22:85–105.
Ocumpaugh, J. (2015). Baker rodrigo ocumpaugh monitoring protocol (bromp) 2.0 technical and training manual. New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences.
Paper, D. (2021). Introduction to Deep Learning, pages 1–24. Apress, Berkeley, CA.
Pekrun, R. (2011). Emotions as drivers of learning and cognitive development. In New perspectives on affect and learning technologies, pages 23–39. Springer.
Picard, R. W. (2010). Affective computing: From laughter to ieee. IEEE Transactions on Affective Computing, 1:11–17.
Raschka, S. (2018). Model evaluation, model selection, and algorithm selection in machine learning.
Rodrigo, M. M. T., Baker, R. S. J., and Nabos, J. Q. (2010). The relationships between sequences of affective states and learner achievement. Proceedings of the 18th International Conference on Computers in Education, pages 56–60.
Scherer, K. R. (2005). What are emotions? and how can they be measured? Social science information, 44(4):695–729.
Shwartz-Ziv, R. and Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion, 81:84–90.
Silvia, P. J. (2010). Confusion and interest: The role of knowledge emotions in aesthetic experience. Psychology of Aesthetics, Creativity, and the Arts, 4:75–80.
Thi, H. A. L., Nguyen, V. V., and Ouchani, S. (2008). Gene selection for cancer classification using dca. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5139 LNAI:62–72.
Tiam-Lee, T. J. and Sumi, K. (2018). Adaptive feedback based on student emotion in a system for programming practice. In Nkambou, R., Azevedo, R., and Vassileva, J., editors, Intelligent Tutoring Systems, pages 243–255, Cham. Springer International Publishing.
Tiam-Lee, T. J. and Sumi, K. (2019). Analysis and prediction of student emotions while doing programming exercises. In International conference on intelligent tutoring systems, pages 24–33. Springer.
Vea, L. and Rodrigo, M. M. (2017). Modeling negative affect detector of novice programming students using keyboard dynamics and mouse behavior. In Numao, M., Theeramunkong, T., Supnithi, T., Ketcham, M., Hnoohom, N., and Pramkeaw, P., editors, Trends in Artificial Intelligence: PRICAI 2016 Workshops, pages 127–138, Cham. Springer International Publishing.
Wong, T. T. (2015). Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, 48:2839–2846.
Yang, T.-Y., Baker, R. S., Studer, C., Heffernan, N., and Lan, A. S. (2019). Active learning for student affect detection. International Educational Data Mining Society.
Arroyo, I., Cooper, D., Burleson, W., Woolf, B., Muldner, K., and Christopherson, R. (2009). Emotion sensors go to school. In Frontiers in Artificial Intelligence and Applications, number 1 in Frontiers in Artificial Intelligence and Applications, pages 17–24. IOS Press.
Badrinath, A., Wang, F., and Pardos, Z. (2021). pybkt: An accessible python library of bayesian knowledge tracing models. International Educational Data Mining Society.
Bosch, N., Chen, Y., and D’Mello, S. (2014). It’s written on your face: Detecting affective states from facial expressions while learning computer programming. In TrausanMatu, S., Boyer, K. E., Crosby, M., and Panourgia, K., editors, Intelligent Tutoring Systems, pages 39–44, Cham. Springer International Publishing.
Bosch, N. and D’Mello, S. (2017). The affective experience of novice computer programmers. International Journal of Artificial Intelligence in Education, 27(1):181–206.
Bosch, N., D’Mello, S., and Mills, C. (2013). What emotions do novices experience during their first computer programming learning session? In International Conference on Artificial Intelligence in Education, pages 11–20. Springer.
Botelho, A. F., Baker, R. S., and Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In André, E., Baker, R., Hu, X., Rodrigo, M. M. T., and du Boulay, B., editors, Artificial Intelligence in Education, pages 40–51, Cham. Springer International Publishing.
Breiman, L. (2001). Random forests. Machine Learning, 45:5–32.
Chen, T. and Guestrin, C. (2016). Xgboost. volume 42, pages 785–794. ACM.
Chi, M. T. and Ohlsson, S. (2005). Complex Declarative Learning. Cambridge University Press.
Coto, M., Mora, S., Grass, B., and Murillo-Morera, J. (2021). Emotions and programming learning: systematic mapping. Computer Science Education, pages 1–36.
D’Mello, S., Person, N., and Lehman, B. (2009). Antecedent-consequent relationships and cyclical patterns between affective states and problem solving outcomes. Frontiers in Artificial Intelligence and Applications, 200:57–64.
D’Mello, S. K. (2020). Big data in the science of learning. In Big data in psychological research., pages 203–225. American Psychological Association.
D’Mello, S., Lehman, B., Pekrun, R., and Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29:153 – 170.
D’Mello, S. K. and Graesser, A. C. (2014). Confusion. In International handbook of emotions in education, pages 299–320. Routledge.
Felipe, D. A. M., Gutierrez, K. I. N., Quiros, E. C. M., and Vea, L. A. (2012). Towards the development of intelligent agent for novice c/c++ programmers through affective analysis of event logs. In Proc. Int. MultiConference Eng. Comput. Sci, volume 1, page 2012. Citeseer.
Graesser, A., Chipman, P., King, B., McDaniel, B., and D’Mello, S. (2007). Emotions and learning with autotutor. In Proceedings of the 2007 Conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, page 569–571, NLD. IOS Press.
Grafsgaard, J. F., Boyer, K. E., and Lester, J. C. (2011). Predicting facial indicators of confusion with hidden markov models. In D’Mello, S., Graesser, A., Schuller, B., and Martin, J.-C., editors, Affective Computing and Intelligent Interaction, pages 97–106, Berlin, Heidelberg. Springer Berlin Heidelberg.
Kubat, M. (2017). An introduction to machine learning, volume 2. Springer.
Kölling, M., Quig, B., Patterson, A., and Rosenberg, J. (2003). The bluej system and its pedagogy. Computer Science Education, 13:249–268.
Lee, D. M. C., Rodrigo, M. M. T., d Baker, R. S., Sugay, J. O., and Coronel, A. (2011). Exploring the relationship between novice programmer confusion and achievement. In International Conference on Affective Computing and Intelligent Interaction, pages 175–184. Springer.
Lehman, B., D’Mello, S., Strain, A., Mills, C., Gross, M., Dobbins, A., Wallace, P., Millis, K., and Graesser, A. (2013). Inducing and tracking confusion with contradictions during complex learning. International Journal of Artificial Intelligence in Education, 22:85–105.
Ocumpaugh, J. (2015). Baker rodrigo ocumpaugh monitoring protocol (bromp) 2.0 technical and training manual. New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences.
Paper, D. (2021). Introduction to Deep Learning, pages 1–24. Apress, Berkeley, CA.
Pekrun, R. (2011). Emotions as drivers of learning and cognitive development. In New perspectives on affect and learning technologies, pages 23–39. Springer.
Picard, R. W. (2010). Affective computing: From laughter to ieee. IEEE Transactions on Affective Computing, 1:11–17.
Raschka, S. (2018). Model evaluation, model selection, and algorithm selection in machine learning.
Rodrigo, M. M. T., Baker, R. S. J., and Nabos, J. Q. (2010). The relationships between sequences of affective states and learner achievement. Proceedings of the 18th International Conference on Computers in Education, pages 56–60.
Scherer, K. R. (2005). What are emotions? and how can they be measured? Social science information, 44(4):695–729.
Shwartz-Ziv, R. and Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion, 81:84–90.
Silvia, P. J. (2010). Confusion and interest: The role of knowledge emotions in aesthetic experience. Psychology of Aesthetics, Creativity, and the Arts, 4:75–80.
Thi, H. A. L., Nguyen, V. V., and Ouchani, S. (2008). Gene selection for cancer classification using dca. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5139 LNAI:62–72.
Tiam-Lee, T. J. and Sumi, K. (2018). Adaptive feedback based on student emotion in a system for programming practice. In Nkambou, R., Azevedo, R., and Vassileva, J., editors, Intelligent Tutoring Systems, pages 243–255, Cham. Springer International Publishing.
Tiam-Lee, T. J. and Sumi, K. (2019). Analysis and prediction of student emotions while doing programming exercises. In International conference on intelligent tutoring systems, pages 24–33. Springer.
Vea, L. and Rodrigo, M. M. (2017). Modeling negative affect detector of novice programming students using keyboard dynamics and mouse behavior. In Numao, M., Theeramunkong, T., Supnithi, T., Ketcham, M., Hnoohom, N., and Pramkeaw, P., editors, Trends in Artificial Intelligence: PRICAI 2016 Workshops, pages 127–138, Cham. Springer International Publishing.
Wong, T. T. (2015). Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, 48:2839–2846.
Yang, T.-Y., Baker, R. S., Studer, C., Heffernan, N., and Lan, A. S. (2019). Active learning for student affect detection. International Educational Data Mining Society.
Published
2022-11-16
How to Cite
KAUTZMANN, Tiago R.; RAMOS, Gabriel de O.; JAQUES, Patrícia A..
The use of student knowledge estimates in computer programming in sensor-free detection models of confusion emotion. In: BRAZILIAN SYMPOSIUM ON COMPUTERS IN EDUCATION (SBIE), 33. , 2022, Manaus.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 1196-1208.
DOI: https://doi.org/10.5753/sbie.2022.225768.
