Quebrando o Silêncio Pedagógico: O Modelo WOKDEX para Feedback Formativo em Juízes Online
Resumo
Juízes online de programação operam sob "silêncio pedagógico": seus vereditos binários (ex: WA, TLE) não orientam o estudante nem distinguem lógicas corretas de aprovações acidentais. Este artigo apresenta o WOKDEX, um schema de metadados que organiza casos de teste com intenção formativa explícita. O modelo propõe uma tipologia de testes que separa exemplos (SAMPLE), testes funcionais (FUNCTIONAL), detecção de misconceptions (MISCONCEPTION) e análise de complexidade (PERFORMANCE). A Heurística de Modulação de Dica (HMD) modula o feedback com base na consistência dos erros. Aplicado a 45 exercícios curados, o WOKDEX demonstra como embutir a intenção pedagógica diretamente no artefato do exercício, preenchendo uma lacuna crítica dos formatos de empacotamento atuais.
Palavras-chave:
Metadados Pedagógicos, Juízes Online, Feedback Formativo, Ensino de Programação, Tipologia de Testes
Referências
ACM/IEEE (2013). Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM Press and IEEE Computer Society Press, New York, NY, USA.
ACM/IEEE (2020). Computing curricula 2020 (cc2020): Paradigms for global computing education. [link].
ADL Initiative (2004). SCORM 2004 4th edition overview. Technical report, Advanced Distributed Learning (ADL) Co-Lab.
ADL Initiative (2015). Experience API (xAPI) specification, version 1.0.3. Technical report, Advanced Distributed Learning (ADL) Co-Lab.
Alves, F. and Jaques, P. A. (2014). Um ambiente virtual com feedback personalizado para apoio a disciplinas de programação. In Anais do XXV Simpósio Brasileiro de Informática na Educação (SBIE), pages 1078–1082. SBC.
Anderson, J. R. (1983). The Architecture of Cognition. Number 5 in Cognitive Science Series. Harvard University Press, Cambridge, MA.
Beecrowd (2024). Beecrowd: Plataforma de problemas de programação. [link].
Black, P. and Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1):7–74.
Costa, E., Fechine, J., Silva, P., and Rocha, H. J. B. (2016). Modelos de Feedback para estudantes em Ambientes Virtuais de Aprendizagem, pages 1–38. DOI: 10.5753/sbc.11436.9.1.
de Campos, C. P. and Ferreira, C. E. (2004). Boca: um sistema de apoio a competições de programação. Journal of Educational Technology, 1(1):25–35.
Edwards, S. H. (2004). Using software testing to move students from trial-and-error to reflection-in-action. ACM SIGCSE Bulletin, 36(1):26–30.
Edwards, S. H. and Pérez-Quiñones, M. A. (2008). Web-cat: Automatically grading programming assignments. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), page 328. ACM.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1):64–74.
ICPC (2024). Problem package format specification. 2026-03-22 [link].
Ihantola, P., Ahoniemi, T., Karavirta, V., and Seppälä, O. (2010). Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling '10, page 86–93, New York, NY, USA. Association for Computing Machinery.
IMS Global Learning Consortium (2013). IMS question and test interoperability (QTI) specification, version 2.1. Technical report, IMS Global Learning Consortium.
IMS Global Learning Consortium (2019). IMS learning tools interoperability (LTI) core specification, version 1.3. Technical report, IMS Global Learning Consortium.
Kattis (2016). Kattis problem archive. [link].
Keuning, H., Jeuring, J., and Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Trans. Comput. Educ., 19(1).
LeetCode (2015). LeetCode: Online coding challenges. [link].
Lobb, R. and Harlow, J. (2016). Coderunner: a tool for assessing computer programming skills. ACM Inroads, 7(1):47–51.
Mariani, J. E., Gerber, N., and Kinkhorst, T. (2005). DOMjudge: An automated judge system for programming contests.
Messer, M., Brown, N. C. C., Kölling, M., and Shi, M. (2024). Automated grading and feedback tools for programming education: A systematic review. ACM Trans. Comput. Educ., 24(1).
Mirzayanov, M., Pavlova, O., Mavrin, P., Melnikov, R., Plotnikov, A., Parfenov, V., and Stankevich, A. (2020). Codeforces as an educational platform for learning programming in digitalization. Olympiads in Informatics, 14:133–142.
Paiva, J. C., Leal, J. P., and Figueira, Á. (2022). Automated assessment in computer science education: A state-of-the-art review. ACM Transactions on Computing Education (TOCE), 22(3):1–40.
Queirós, R. and Leal, J. P. (2012). PExIL: XML language for programming exercises interoperability. Informatics in Education, 11(1):95–112.
Robins, A. (2019). Novice programmers and introductory programming. The Cambridge Handbook of Computing Education Research, pages 327–376.
Robins, A., Rountree, J., and Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2):137–172.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational researcher, 15(2):4–14.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1):153–189.
Silva, E. P. d. (2024). Misconceptions in Correct Code: Assisting Instructors and Students by Shedding Light on What is Potentially Overshadowed by Automated Correction. Tese de doutorado, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brasil.
Striewe, M. and Balz, M. (2015). A standard format for programming exercises. Formative Assessment.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive science, 12(2):257–285.
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA.
Wasik, S., Antczak, M., Badura, J., Laskowski, A., and Sternal, T. (2018). A survey on online judge systems and their applications. ACM Comput. Surv., 51(1).
Watson, C. and Li, F. W. (2014). Failure rates in introductory programming revisited. ACM Transactions on Computing Education, 14(1):1–24.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering. Springer.
ACM/IEEE (2020). Computing curricula 2020 (cc2020): Paradigms for global computing education. [link].
ADL Initiative (2004). SCORM 2004 4th edition overview. Technical report, Advanced Distributed Learning (ADL) Co-Lab.
ADL Initiative (2015). Experience API (xAPI) specification, version 1.0.3. Technical report, Advanced Distributed Learning (ADL) Co-Lab.
Alves, F. and Jaques, P. A. (2014). Um ambiente virtual com feedback personalizado para apoio a disciplinas de programação. In Anais do XXV Simpósio Brasileiro de Informática na Educação (SBIE), pages 1078–1082. SBC.
Anderson, J. R. (1983). The Architecture of Cognition. Number 5 in Cognitive Science Series. Harvard University Press, Cambridge, MA.
Beecrowd (2024). Beecrowd: Plataforma de problemas de programação. [link].
Black, P. and Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1):7–74.
Costa, E., Fechine, J., Silva, P., and Rocha, H. J. B. (2016). Modelos de Feedback para estudantes em Ambientes Virtuais de Aprendizagem, pages 1–38. DOI: 10.5753/sbc.11436.9.1.
de Campos, C. P. and Ferreira, C. E. (2004). Boca: um sistema de apoio a competições de programação. Journal of Educational Technology, 1(1):25–35.
Edwards, S. H. (2004). Using software testing to move students from trial-and-error to reflection-in-action. ACM SIGCSE Bulletin, 36(1):26–30.
Edwards, S. H. and Pérez-Quiñones, M. A. (2008). Web-cat: Automatically grading programming assignments. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), page 328. ACM.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1):64–74.
ICPC (2024). Problem package format specification. 2026-03-22 [link].
Ihantola, P., Ahoniemi, T., Karavirta, V., and Seppälä, O. (2010). Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling '10, page 86–93, New York, NY, USA. Association for Computing Machinery.
IMS Global Learning Consortium (2013). IMS question and test interoperability (QTI) specification, version 2.1. Technical report, IMS Global Learning Consortium.
IMS Global Learning Consortium (2019). IMS learning tools interoperability (LTI) core specification, version 1.3. Technical report, IMS Global Learning Consortium.
Kattis (2016). Kattis problem archive. [link].
Keuning, H., Jeuring, J., and Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Trans. Comput. Educ., 19(1).
LeetCode (2015). LeetCode: Online coding challenges. [link].
Lobb, R. and Harlow, J. (2016). Coderunner: a tool for assessing computer programming skills. ACM Inroads, 7(1):47–51.
Mariani, J. E., Gerber, N., and Kinkhorst, T. (2005). DOMjudge: An automated judge system for programming contests.
Messer, M., Brown, N. C. C., Kölling, M., and Shi, M. (2024). Automated grading and feedback tools for programming education: A systematic review. ACM Trans. Comput. Educ., 24(1).
Mirzayanov, M., Pavlova, O., Mavrin, P., Melnikov, R., Plotnikov, A., Parfenov, V., and Stankevich, A. (2020). Codeforces as an educational platform for learning programming in digitalization. Olympiads in Informatics, 14:133–142.
Paiva, J. C., Leal, J. P., and Figueira, Á. (2022). Automated assessment in computer science education: A state-of-the-art review. ACM Transactions on Computing Education (TOCE), 22(3):1–40.
Queirós, R. and Leal, J. P. (2012). PExIL: XML language for programming exercises interoperability. Informatics in Education, 11(1):95–112.
Robins, A. (2019). Novice programmers and introductory programming. The Cambridge Handbook of Computing Education Research, pages 327–376.
Robins, A., Rountree, J., and Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2):137–172.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational researcher, 15(2):4–14.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1):153–189.
Silva, E. P. d. (2024). Misconceptions in Correct Code: Assisting Instructors and Students by Shedding Light on What is Potentially Overshadowed by Automated Correction. Tese de doutorado, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brasil.
Striewe, M. and Balz, M. (2015). A standard format for programming exercises. Formative Assessment.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive science, 12(2):257–285.
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA.
Wasik, S., Antczak, M., Badura, J., Laskowski, A., and Sternal, T. (2018). A survey on online judge systems and their applications. ACM Comput. Surv., 51(1).
Watson, C. and Li, F. W. (2014). Failure rates in introductory programming revisited. ACM Transactions on Computing Education, 14(1):1–24.
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in Software Engineering. Springer.
Publicado
05/10/2026
Como Citar
MIRANDA JUNIOR, Alessio; DOS SANTOS, Vinicius Fernandes.
Quebrando o Silêncio Pedagógico: O Modelo WOKDEX para Feedback Formativo em Juízes Online. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 37. , 2026, Goiânia/GO.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 1886-1900.
DOI: https://doi.org/10.5753/sbie.2026.28122.
