From Importance to Action: Evaluating Counterfactual Explanations under Constraints in Brazilian Public Schools

• Ana Beatriz P. de A. Rebouças UFC
• César L. C. Mattos UFC

DOI: https://doi.org/10.5753/eniac.2026.31183

Resumo

Machine learning has been increasingly used to predict educational outcomes. However, prediction alone is insufficient for meaningful intervention, where the main goal is not to label students or schools, but to identify actionable paths toward improvement. Counterfactual Explanations (CFEs) address this by identifying minimal feature changes associated with better outcomes. In large-scale exams such as Enem, the student cohort changes each year, limiting individual-level analysis, while municipality-level aggregation introduces substantial heterogeneity. Hence, we analyze CFEs at school level, combining Enem 2024 results with the 2024 Brazilian School Census. Focusing on a realistic policy setting, we restrict the analysis to public state schools and incorporate intervention constraints through different feature mutability levels. First, we train five classifiers to predict school performance. Then, we generate and evaluate CFEs for the best-performing model (XGBoost). We compare results across mutability constraint levels, generation methods (DiCE, Growing Spheres, and FACE), and school subgroups. Our results reveal actionability discrepancies across subgroups under operational constraints, with implications for equitable interventions. We also contrast SHAP feature importance with CFEs, revealing a gap between predictive importance and actionable change, which is critical for distinguishing explanatory relevance from intervention-oriented insights.

Referências

Afrin, F., Hamilton, M., and Thevathyan, C. (2023). Exploring counterfactual explanations for predicting student success. In ICCS 2023, pages 413–420. Springer.

Bayrak, B. and Bach, K. (2024). Evaluation of instance-based explanations: an in-depth analysis of counterfactual evaluation metrics, challenges, and the ceval toolkit. IEEE Access, 12:137683–137695.

Buñay-Guisñan et al. (2025). Group counterfactual explanations: A use case to support students at risk of dropping out in online education. Electronics, 15(1):51.

Buñay-Guisñan, P., Lara, J., and Romero, C. (2026). Counterfactual explanations in education: A systematic review. WIREs Data Mining and Knowledge Discovery, 16(1):e70060.

Cavus, M. and Kuzilek, J. (2024). The actionable explanations for student success prediction models: A benchmark study on the quality of counterfactual methods. In EDM.

de Jesus, A. C. et al. (2025). Understanding public school enrollment trends in the northeast of brazil: Unraveling features through ai related to the learning process for low-income students. In AIED 2025, pages 454–461. Springer.

Dutra, J. F., Júnior, J. B. F., and de Souza Fernandes, D. Y. (2023). Fatores que podem interferir no desempenho de estudantes no enem: uma revisão sistemática da literatura. Revista Brasileira de Informática na Educação, 31:323–351.

Guidotti, R. (2024). Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining and Knowledge Discovery, 38(5):2770–2824.

Karimi, A.-H., Schölkopf, B., and Valera, I. (2021). Algorithmic recourse: from counterfactual explanations to interventions. In ACM FAccT 2021, pages 353–362.

Khosravi, H. et al. (2022). Explainable artificial intelligence in education. Computers and education: artificial intelligence, 3:100074.

Kumar, I. E. et al. (2020). Problems with shapley-value-based explanations as feature importance measures. In ICML 2020, pages 5491–5500. PMLR.

Laugel, T. et al. (2018). Comparison-based inverse classification for interpretability in machine learning. In IPMU 2018, pages 100–111. Springer.

Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In NeurIPS 2017.

Melo, R. O. et al. (2021). Impacto das variáveis socioeconômicas no desempenho do enem: uma análise espacial e sociológica. Rev. de Admin. Pública, 55(6):1271–1294.

Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38.

Moreira, C. et al. (2025). Benchmarking instance-centric counterfactual algorithms for xai: from white box to black box. ACM Computing Surveys, 57(6):1–37.

Mothilal, R. et al. (2021). Towards unifying feature attribution and counterfactual explanations: Different means to the same end. In AAAI/ACM AIES 2021, pages 652–663.

Mothilal, R. K., Sharma, A., and Tan, C. (2020). Explaining machine learning classifiers through diverse counterfactual explanations. In ACM FAccT 2020, pages 607–617.

Poyiadzi, R. et al. (2020). Face: feasible and actionable counterfactual explanations. In AAAI/ACM AIES, pages 344–350.

Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ” why should i trust you?” explaining the predictions of any classifier. In ACM SIGKDD 2016, pages 1135–1144.

Silva Filho, R. L. C., Brito, K., and Adeodato, P. J. L. (2023). Leveraging causal reasoning in educational data mining: an analysis of brazilian secondary education. Applied Sciences, 13(8):5198.

Suaza-Medina, M., Peñabaena-Niebles, R., and Jubiz-Diaz, M. (2024). A model for predicting academic performance on standardised tests for lagging regions based on machine learning and shapley additive explanations. Scientific Reports, 14(1):25306.

Swamy, V., Du, S., Marras, M., and Kaser, T. (2023). Trusting the explainers: teacher validation of explainable artificial intelligence for course design. In LAK 2023.

Tsiakmaki, M. and Ragos, O. (2021). A case study of interpretable counterfactual explanations for the task of predicting student academic performance. In CSCC 2021.

Ustun, B., Spangher, A., and Liu, Y. (2019). Actionable recourse in linear classification. In ACM FAccT 2019, pages 10–19.

Verma, S. et al. (2024). Counterfactual explanations and algorithmic recourses for machine learning: A review. ACM Computing Surveys, 56(12):1–42.

Wachter, S., Mittelstadt, B., and Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841.