Explainability of models for Automatic Essay Assessment

• Gabriel Menezes Moreira UFPI
• Raimundo Santos Moura UFPI

DOI: https://doi.org/10.5753/enucompi.2023.26624

Abstract

In the context of Automated Essay Scoring (AES), one of the main learning points for the student is to receive feedback on their correction to understand where and what they have done wrong. In this sense, this work aims to research interpretability methods for AI models, with a focus on AES activity in the style of the National High School Exam (ENEM, Exame Nacional do Ensino Médio). The achieved results reveal that LIME (Local Interpretable Model-Agnostic Explanations) highlights crucial terms and trends that the model associates with in the AES task. These particularities played a fundamental role in identifying the strengths and weaknesses of the developed models.

Keywords: LIME, AAR, LSTM, Explainability, AI, AM

References

Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater® V. 2. Disponível em: [link].

Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32. Disponível em: [link].

Doshi-Velez, F., & Kim, B. (2017). Towards a Rigorous Science of Interpretable Machine Learning. Disponível em: [link].

Kingma, D. P. and Ba, J. (2015). Adam: A method for stochastic optimization. Disponível em: [link].

Lipton, Z. C. (2018). The mythos of model interpretability. Disponível em: [link].

Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Disponível em: [link].

Marinho, J. C., Anchiêta, R. T., Moura, R. S. (2022) Essay-BR: a Brazilian Corpus to Automatic Essay Scoring Task. Disponível em: [link].

Marinho, Jeziel C.; Cordeiro, Fábio; Anchiêta, Rafael T.; Moura, Raimundo S. (2022) Automated Essay Scoring: An approach based on ENEM competencies. Disponível em: [link].

MEC. (2019). Cartilha do INEP. Acesso em 04-08-2023. Disponível em: [link].

Molnar, C., Casalicchio, G., & Bischl, B. (2021). Interpretable machine learning–a brief history, state-of-the-art and challenges. Disponível em: [link].

Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019). Interpretable machine learning: definitions, methods, and applications. Disponível em: [link].

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Disponível em: [link].

Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Disponível em: [link].