When the Split Matters: Reorganizing Micro-Edited Pairs for Portuguese Pun Detection

• Avelar Rodrigues de Sousa USP
• Maykon Willyam de Sousa Ferreira USP
• Victor Oliveira Lima Pontes USP
• Rafael Torres Anchiêta IFMA

DOI: https://doi.org/10.5753/eniac.2026.31369

Resumo

Pun detection is a challenging task in Natural Language Processing, as it relies on lexical, semantic, phonological, and contextual cues. The Puntuguese corpus contains micro-edited pairs of punning and non-punning sentences that share the same base identifier, which may create dependencies when pairs are distributed across training, validation, and test splits. This paper investigates how split organization affects pun detection evaluation in Portuguese. We compare the original Puntuguese splits with two controlled organizations: Pair-Preserving, which keeps paired instances in the same partition, and Maximum-Crossing, which maximizes pair distribution across partitions. We evaluate a BERTimbau-based classifier and an ensemble of traditional classifiers. Results show that split organization substantially affects model performance and ranking: the ensemble benefits from increasing pair crossing, with accuracy varying from 0.47 in Pair-Preserving to 0.80 in the Original split and 0.96 in Maximum-Crossing, whereas BERTimbau large achieves its highest accuracy in the Pair-Preserving setting (0.76) and does not benefit from additional crossing. Consequently, conclusions drawn from the original splits do not hold once the pairing is controlled, and we recommend that evaluations on Puntuguese report and, when appropriate, control the paired structure of the corpus.

Referências

Anchiêta, R. T., Santos, R. L. d. S., and Moura, R. S. (2026). Token-level pun location using multi-layer BERT with mixture of experts. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 260–269, Salvador, Brazil. Association for Computational Linguistics.

Attardo, S. (2020). The Linguistics of Humor: An Introduction. Oxford University Press, Oxford.

Hossain, N., Krumm, J., and Gamon, M. (2019). “president vows to cut hair”: Dataset and analysis of creative text editing for humorous headlines. In Burstein, J., Doran, C., and Solorio, T., editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 133–142, Minneapolis, Minnesota. Association for Computational Linguistics.

Inacio, M. L., Wick-Pedro, G., Ramisch, R., Espírito Santo, L., Chacon, X. S. Q., Santos, R., Sousa, R., Anchiêta, R., and Goncalo Oliveira, H. (2024). Puntuguese: A corpus of puns in Portuguese with micro-edits. In Calzolari, N., Kan, M.-Y., Hoste, V., Lenci, A., Sakti, S., and Xue, N., editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13332–13343, Torino, Italia. ELRA and ICCL.

Kalloniatis, A. and Adamidis, P. (2024). Computational humor recognition: A systematic literature review. Artificial Intelligence Review, 58(2):43.

Kao, J. T., Levy, R., and Goodman, N. D. (2016). A computational model of linguistic humor in puns. Cognitive Science, 40(5):1270–1285.

Kaufman, S., Rosset, S., Perlich, C., and Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4).

Leal, J., Inácio, M., Oliveira, H., and Anchiêta, R. (2025). Improving pun detection with an ensemble of traditional machine learning methods. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 206–219, Porto Alegre, RS, Brasil. SBC.

Miller, T., Hempelmann, C., and Gurevych, I. (2017). SemEval-2017 task 7: Detection and interpretation of English puns. In Bethard, S., Carpuat, M., Apidianaki, M., Mohammad, S. M., Cer, D., and Jurgens, D., editors, Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 58–68, Vancouver, Canada. Association for Computational Linguistics.

Sousa, A. R. d., Sousa, C. S., Barros, C. H. S., and Anchiêta, R. T. (2026). Extending an ensemble baseline with corpus-based graph features for Portuguese pun detection. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1, pages 759–769, Salvador, Brazil. Association for Computational Linguistics.

Sousa, Ê., Sousa, A., and Anchiêta, R. (2025). Classificação de trocadilhos em português com BERTimbau large: Desafios e resultados. In Anais do XVII Encontro Unificado de Computação do Piauí, pages 139–148, Porto Alegre, RS, Brasil. SBC.

Souza, F., Nogueira, R., and Lotufo, R. (2020). BERTimbau: Pretrained BERT models for Brazilian Portuguese. In Cerri, R. and Prati, R. C., editors, Intelligent Systems: 9th Brazilian Conference on Intelligent Systems (BRACIS 2020), volume 12319 of Lecture Notes in Computer Science, pages 403–417, Cham. Springer.