Improving Pun Detection with an Ensemble of Traditional Machine Learning Methods
Abstract
Humor is a remarkably complex emotional process, defined as any object or event that causes laughter or amusement or is considered funny. Therefore, recognizing humor is considered one of the most challenging tasks in Natural Language Processing. In this paper, we approached the pun detection task for the Portuguese language. Puns are a form of wordplay that exploits multiple meanings of a term or similar-sounding words to create an intended humorous or rhetorical effect. Our strategy is straightforward: we trained and evaluated an ensemble learning approach of traditional machine learning models on PUNTUGUESE, a recent corpus of Portuguese puns. With this, we outperformed a BERT-based model by 11 p.p. in accuracy and achieved state-of-the-art results. More than that, we performed a detailed error analysis and found that our approach has limitations in identifying puns that contain neologisms.References
Anchiêta, R. T., Neto, F. A. R., Marinho, J. C., do Nascimento, K. V., and Moura, R. S. (2021). Piln idpt 2021: Irony detection in portuguese texts with superficial features and embeddings. In IberLEF@ SEPLN, pages 917–924, Málaga, Spain. CEUR.
Attardo, S. (2020). The linguistics of humor: An introduction. Oxford University Press.
Banas, J. A., Dunbar, N., Rodriguez, D., and Liu, S.-J. (2011). A review of humor in educational settings: Four decades of research. Communication Education, 60(1):115–144.
Bird, S., Klein, E., and Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”.
Carvalho, P., Martins, B., Rosa, H., Amir, S., Baptista, J., and Silva, M. J. (2020). Situational irony in farcical news headlines. In International Conference on Computational Processing of the Portuguese Language, pages 65–75, Evora, Portugual. Springer.
Castro, S., Chiruzzo, L., and Rosá, A. (2018). Overview of the haha task: Humor analysis based on human annotation at ibereval 2018. In IberEval SEPLN, pages 187–194.
Clark, K., Luong, M., Le, Q. V., and Manning, C. D. (2020). ELECTRA: pre-training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, Addis Ababa, Ethiopia. Curran Associates, Inc.
Clemêncio, A. D. F. (2019). Reconhecimento automático de humor verbal. Master’s thesis, Universidade de Coimbra.
Corrêa, U. B., Coelho, L., Santos, L., and de Freitas, L. A. (2021). Overview of the IDPT task on irony detection in portuguese at IberLEF 2021. Procesamiento del Lenguaje Natural, 67:269–276.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Diao, Y., Lin, H., Wu, D., Yang, L., Xu, K., Yang, Z., Wang, J., Zhang, S., Xu, B., and Zhang, D. (2018). WECA: A WordNet-encoded collocation-attention network for homographic pun recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2507–2516, Brussels, Belgium. Association for Computational Linguistics.
Diao, Y., Lin, H., Yang, L., Fan, X., Wu, D., Zhang, D., and Xu, K. (2019). Heterographic pun recognition via pronunciation and spelling understanding gated attention network. In The World Wide Web Conference, pages 363–371, San Francisco, CA, USA. Association for Computing Machinery.
Ermakova, L., Bosser, A.-G., Miller, T., Thomas, T., Preciado, V. M. P., Sidorov, G., and Jatowt, A. (2024). Clef 2024 joker lab: Automatic humour analysis. In European Conference on Information Retrieval, pages 36–43, Glasgow, UK. Springer.
Ermakova, L., Miller, T., Regattin, F., Bosser, A.-G., Borg, C., Mathurin, É., Le Corre, G., Araújo, S., Hannachi, R., Boccou, J., et al. (2022). Overview of joker@ clef 2022: automatic wordplay and humour translation workshop. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 447–469, Bologna, Italy. Springer.
Gameiro, P., Inácio, M. L., Gonçalo Oliveira, H., and Alves, A. (2024). Sequence labeling for pun location and detection in portuguese. In EPIA Conference on Artificial Intelligence, pages 254–266, Viana do Castelo, Portugal. Springer.
Gonçalo Oliveira, H., Clemêncio, A., and Alves, A. (2020). Corpora and baselines for humour recognition in Portuguese. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1278–1285, Marseille, France. European Language Resources Association.
He, P., Gao, J., and Chen, W. (2023). Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations, Kigali, Rwanda. Curran Associates, Inc.
Inácio, M. L. and Gonçalo Oliveira, H. (2024). Exploring multimodal models for humor recognition in Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese Vol. 1, pages 568–574, Santiago de Compostela, Galicia/Spain. Association for Computational Linguistics.
Inacio, M. L., Wick-Pedro, G., Ramisch, R., Espírito Santo, L., Chacon, X. S. Q., Santos, R., Sousa, R., Anchiêta, R., and Goncalo Oliveira, H. (2024). Puntuguese: A corpus of puns in Portuguese with micro-edits. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13332–13343, Torino, Italia. ELRA and ICCL.
Jaiswal, A. and Monika (2019). Pun detection using soft computing techniques. In International conference on machine learning, big data, cloud and parallel computing (COMITCon), pages 5–9, Faridabad, India. IEEE.
Kalloniatis, A. and Adamidis, P. (2024). Computational humor recognition: a systematic literature review. Artificial Intelligence Review, 58(2):43.
Kao, J. T., Levy, R., and Goodman, N. D. (2016). A computational model of linguistic humor in puns. Cognitive science, 40(5):1270–1285.
Labadie Tamayo, R., Chulvi-Ferriols, M. A., and Rosso, P. (2023). Everybody hurts, sometimes overview of hurtful humour at iberlef 2023: Detection of humour spreading prejudice in twitter. Procesamiento del lenguaje natural, 71:383–395.
Lima Inácio, M., Wick-pedro, G., and Goncalo Oliveira, H. (2023). What do humor classifiers learn? an attempt to explain humor recognition models. In Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 88–98, Dubrovnik, Croatia. Association for Computational Linguistics.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, page 4768–4777, Long Beach, California, USA. Curran Associates, Inc.
Luz, A. I., Santos, H., Medeiros, M. M., and Anchiêta, R. T. (2023). Improving irony detection by balancing methods and feature selection. In Anais do XII Brazilian Workshop on Social Network Analysis and Mining, pages 216–221, João Pessoa, Brazil. SBC.
Miller, T., Hempelmann, C., and Gurevych, I. (2017). SemEval-2017 task 7: Detection and interpretation of English puns. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 58–68, Vancouver, Canada. Association for Computational Linguistics.
Monika and Vij, S. (2019). Humour agent detection in puns. In 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), pages 1–6, Ghaziabad, India. IEEE.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.
Ren, L., Xu, B., Lin, H., and Yang, L. (2021). Abml: attention-based multi-task learning for jointly humor recognition and pun detection. Soft Computing, 25(22):14109–14118.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, San Francisco, California, USA. Association for Computing Machinery.
Rodrigues, J., Gomes, L., Silva, J., Branco, A., Santos, R., Cardoso, H. L., and Osório, T. (2023). Advancing neural encoding of portuguese with transformer albertina pt-*. In EPIA Conference on Artificial Intelligence, pages 441–453, Faial Island, Azores. Springer.
Sammut, C. and Webb, G. I. (2011). Encyclopedia of machine learning. Springer Science & Business Media.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: pretrained bert models for brazilian portuguese. In Proceedings of the 9th Brazilian Conference on Intelligent Systems (BRACIS), pages 403–417, Rio Grande do Sul, Brazil. Springer.
Wick-Pedro, G., da Silva, C. F., Inácio, M. L., Vale, O. A., and de Medeiros Caseli, H. (2024). Using large language models for identifying satirical news in Brazilian Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese, pages 156–167, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Wick-Pedro, G. and Santos, R. L. (2021). Complexidade textual em notícias satíricas: uma análise para o português do brasil. In Anais do XIII Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 409–415, Online. SBC.
Xu, Z., Yuan, S., Chen, L., and Yang, D. (2024). “a good pun is its own reword”: Can large language models understand puns? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11766–11782, Miami, Florida, USA. Association for Computational Linguistics.
Yang, D., Lavie, A., Dyer, C., and Hovy, E. (2015). Humor recognition and humor anchor extraction. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2367–2376, Lisbon, Portugal. Association for Computational Linguistics.
Yatsu, M. and Araki, K. (2018). Comparison of pun detection methods using Japanese pun corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3602 – 3605, Miyazaki, Japan. European Language Resources Association (ELRA).
Zhou, Y., Jiang, J.-Y., Zhao, J., Chang, K.-W., and Wang, W. (2020). “the boating store had its best sail ever”: Pronunciation-attentive contextualized pun recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 813–822, Online. Association for Computational Linguistics.
Zou, Y. and Lu, W. (2019). Joint detection and location of English puns. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2117–2123, Minneapolis, Minnesota. Association for Computational Linguistics.
Attardo, S. (2020). The linguistics of humor: An introduction. Oxford University Press.
Banas, J. A., Dunbar, N., Rodriguez, D., and Liu, S.-J. (2011). A review of humor in educational settings: Four decades of research. Communication Education, 60(1):115–144.
Bird, S., Klein, E., and Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”.
Carvalho, P., Martins, B., Rosa, H., Amir, S., Baptista, J., and Silva, M. J. (2020). Situational irony in farcical news headlines. In International Conference on Computational Processing of the Portuguese Language, pages 65–75, Evora, Portugual. Springer.
Castro, S., Chiruzzo, L., and Rosá, A. (2018). Overview of the haha task: Humor analysis based on human annotation at ibereval 2018. In IberEval SEPLN, pages 187–194.
Clark, K., Luong, M., Le, Q. V., and Manning, C. D. (2020). ELECTRA: pre-training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, Addis Ababa, Ethiopia. Curran Associates, Inc.
Clemêncio, A. D. F. (2019). Reconhecimento automático de humor verbal. Master’s thesis, Universidade de Coimbra.
Corrêa, U. B., Coelho, L., Santos, L., and de Freitas, L. A. (2021). Overview of the IDPT task on irony detection in portuguese at IberLEF 2021. Procesamiento del Lenguaje Natural, 67:269–276.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Diao, Y., Lin, H., Wu, D., Yang, L., Xu, K., Yang, Z., Wang, J., Zhang, S., Xu, B., and Zhang, D. (2018). WECA: A WordNet-encoded collocation-attention network for homographic pun recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2507–2516, Brussels, Belgium. Association for Computational Linguistics.
Diao, Y., Lin, H., Yang, L., Fan, X., Wu, D., Zhang, D., and Xu, K. (2019). Heterographic pun recognition via pronunciation and spelling understanding gated attention network. In The World Wide Web Conference, pages 363–371, San Francisco, CA, USA. Association for Computing Machinery.
Ermakova, L., Bosser, A.-G., Miller, T., Thomas, T., Preciado, V. M. P., Sidorov, G., and Jatowt, A. (2024). Clef 2024 joker lab: Automatic humour analysis. In European Conference on Information Retrieval, pages 36–43, Glasgow, UK. Springer.
Ermakova, L., Miller, T., Regattin, F., Bosser, A.-G., Borg, C., Mathurin, É., Le Corre, G., Araújo, S., Hannachi, R., Boccou, J., et al. (2022). Overview of joker@ clef 2022: automatic wordplay and humour translation workshop. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 447–469, Bologna, Italy. Springer.
Gameiro, P., Inácio, M. L., Gonçalo Oliveira, H., and Alves, A. (2024). Sequence labeling for pun location and detection in portuguese. In EPIA Conference on Artificial Intelligence, pages 254–266, Viana do Castelo, Portugal. Springer.
Gonçalo Oliveira, H., Clemêncio, A., and Alves, A. (2020). Corpora and baselines for humour recognition in Portuguese. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1278–1285, Marseille, France. European Language Resources Association.
He, P., Gao, J., and Chen, W. (2023). Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations, Kigali, Rwanda. Curran Associates, Inc.
Inácio, M. L. and Gonçalo Oliveira, H. (2024). Exploring multimodal models for humor recognition in Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese Vol. 1, pages 568–574, Santiago de Compostela, Galicia/Spain. Association for Computational Linguistics.
Inacio, M. L., Wick-Pedro, G., Ramisch, R., Espírito Santo, L., Chacon, X. S. Q., Santos, R., Sousa, R., Anchiêta, R., and Goncalo Oliveira, H. (2024). Puntuguese: A corpus of puns in Portuguese with micro-edits. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13332–13343, Torino, Italia. ELRA and ICCL.
Jaiswal, A. and Monika (2019). Pun detection using soft computing techniques. In International conference on machine learning, big data, cloud and parallel computing (COMITCon), pages 5–9, Faridabad, India. IEEE.
Kalloniatis, A. and Adamidis, P. (2024). Computational humor recognition: a systematic literature review. Artificial Intelligence Review, 58(2):43.
Kao, J. T., Levy, R., and Goodman, N. D. (2016). A computational model of linguistic humor in puns. Cognitive science, 40(5):1270–1285.
Labadie Tamayo, R., Chulvi-Ferriols, M. A., and Rosso, P. (2023). Everybody hurts, sometimes overview of hurtful humour at iberlef 2023: Detection of humour spreading prejudice in twitter. Procesamiento del lenguaje natural, 71:383–395.
Lima Inácio, M., Wick-pedro, G., and Goncalo Oliveira, H. (2023). What do humor classifiers learn? an attempt to explain humor recognition models. In Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 88–98, Dubrovnik, Croatia. Association for Computational Linguistics.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, page 4768–4777, Long Beach, California, USA. Curran Associates, Inc.
Luz, A. I., Santos, H., Medeiros, M. M., and Anchiêta, R. T. (2023). Improving irony detection by balancing methods and feature selection. In Anais do XII Brazilian Workshop on Social Network Analysis and Mining, pages 216–221, João Pessoa, Brazil. SBC.
Miller, T., Hempelmann, C., and Gurevych, I. (2017). SemEval-2017 task 7: Detection and interpretation of English puns. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 58–68, Vancouver, Canada. Association for Computational Linguistics.
Monika and Vij, S. (2019). Humour agent detection in puns. In 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), pages 1–6, Ghaziabad, India. IEEE.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.
Ren, L., Xu, B., Lin, H., and Yang, L. (2021). Abml: attention-based multi-task learning for jointly humor recognition and pun detection. Soft Computing, 25(22):14109–14118.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, San Francisco, California, USA. Association for Computing Machinery.
Rodrigues, J., Gomes, L., Silva, J., Branco, A., Santos, R., Cardoso, H. L., and Osório, T. (2023). Advancing neural encoding of portuguese with transformer albertina pt-*. In EPIA Conference on Artificial Intelligence, pages 441–453, Faial Island, Azores. Springer.
Sammut, C. and Webb, G. I. (2011). Encyclopedia of machine learning. Springer Science & Business Media.
Souza, F., Nogueira, R., and Lotufo, R. (2020). Bertimbau: pretrained bert models for brazilian portuguese. In Proceedings of the 9th Brazilian Conference on Intelligent Systems (BRACIS), pages 403–417, Rio Grande do Sul, Brazil. Springer.
Wick-Pedro, G., da Silva, C. F., Inácio, M. L., Vale, O. A., and de Medeiros Caseli, H. (2024). Using large language models for identifying satirical news in Brazilian Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese, pages 156–167, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Wick-Pedro, G. and Santos, R. L. (2021). Complexidade textual em notícias satíricas: uma análise para o português do brasil. In Anais do XIII Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 409–415, Online. SBC.
Xu, Z., Yuan, S., Chen, L., and Yang, D. (2024). “a good pun is its own reword”: Can large language models understand puns? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11766–11782, Miami, Florida, USA. Association for Computational Linguistics.
Yang, D., Lavie, A., Dyer, C., and Hovy, E. (2015). Humor recognition and humor anchor extraction. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2367–2376, Lisbon, Portugal. Association for Computational Linguistics.
Yatsu, M. and Araki, K. (2018). Comparison of pun detection methods using Japanese pun corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3602 – 3605, Miyazaki, Japan. European Language Resources Association (ELRA).
Zhou, Y., Jiang, J.-Y., Zhao, J., Chang, K.-W., and Wang, W. (2020). “the boating store had its best sail ever”: Pronunciation-attentive contextualized pun recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 813–822, Online. Association for Computational Linguistics.
Zou, Y. and Lu, W. (2019). Joint detection and location of English puns. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2117–2123, Minneapolis, Minnesota. Association for Computational Linguistics.
Published
2025-09-29
How to Cite
LEAL, Jhúlia de Souza; INÁCIO, Marcio Lima; OLIVEIRA, Hugo Gonçalo; ANCHIÊTA, Rafael Torres.
Improving Pun Detection with an Ensemble of Traditional Machine Learning Methods. In: BRAZILIAN SYMPOSIUM IN INFORMATION AND HUMAN LANGUAGE TECHNOLOGY (STIL), 16. , 2025, Fortaleza/CE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 206-219.
DOI: https://doi.org/10.5753/stil.2025.37826.
