Estimating the Difficulty of Programming Problems in Brazilian Portuguese with Few Examples: An Evaluation of BERTimbau Embeddings - Sena, João Pedro M.; Barbosa, Ellen Francine; Santos, André G.; Reis, Julio C. S. (Anais do Simpósio Brasileiro de Informática na Educação (SBIE), 2025)
Programming practice is a fundamental step in learning to program. In this context, estimating the difficulty of programming questions is crucial for effective adaptive learning. Some approaches address this problem by analyzing students’ past performance, but this can be impractical as sometimes only the problem text is available. For this reason, this study evaluates the use of semantic embeddings of exercise text (QDET) extracted with BERTimbau, a state-of-the-art BERT model pre-trained for Brazilian Portuguese, in a context where only a small sample is available for training, a task referred to as few-shot learning. The results show that the embeddings generated by BERTimbau do not effectively capture the difficulty of problems based solely on textual information. This emphasizes the need for models trained specifically for this task.