Validating Semantic Content in Sentence Simplification using OpenIE and Relational Graphs
Resumo
A simplificação de sentenças frequentemente altera a estrutura textual enquanto preserva o significado, tornando a fidelidade semântica difícil de avaliar. Neste trabalho, investigamos se representações baseadas em grafos derivadas de Extração Aberta de Informação (Open IE) podem capturar a preservação semântica em diferentes níveis de simplificação. Utilizando o corpus PorSimplesSent, extraímos triplas relacionais de sentenças originais e simplificadas para construir grafos semânticos comparáveis. Os resultados fornecem indícios exploratórios de que a análise baseada em grafos pode capturar diferenças semânticas relevantes e fornece sinais complementares além da sobreposição lexical superficial para avaliar a qualidade da simplificação.Referências
Agrawal, S. and Carpuat, M. (2024). Do text simplification systems preserve meaning? a human evaluation via reading comprehension. Transactions of the Association for Computational Linguistics, 12:432–448.
Al-Thanyyan, S. S. and Azmi, A. M. (2021). Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1–36.
Alva-Manchego, F., Scarton, C., and Specia, L. (2020). Data-driven sentence simplification: Survey and benchmark. Computational Linguistics, 46(1):135–187.
Alva-Manchego, F., Scarton, C., and Specia, L. (2021). The (un)suitability of automatic evaluation metrics for text simplification. Computational Linguistics, 47(4):861–889.
Alves, A., Miranda, P., Mello, R., and Nascimento, A. (2023). Automatic simplification of legal texts in portuguese using machine learning. In Legal Knowledge and Information Systems, pages 281–286. IOS Press.
Baez, A. and Saggion, H. (2023). LSLlama: Fine-tuned LLaMA for lexical simplification. In Štajner, S., Saggio, H., Shardlow, M., and Alva-Manchego, F., editors, Proceedings of the Second Workshop on Text Simplification, Accessibility and Readability, pages 102–108, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
Chandrasekar, R., Doran, C., and Bangalore, S. (1996a). Motivations and methods for text simplification. In COLING 1996 volume 2: The 16th international conference on computational linguistics.
Chandrasekar, R., Doran, C., and Srinivas, B. (1996b). Motivations and methods for text simplification. In COLING 1996 Volume 2: The 16th International Conference on Computational Linguistics.
de Menezes, L. C., Paes, A., and Finatto, M. J. B. (2023). Abordagem baseada em aumento de dados para avaliação automática de leiturabilidade. Domínios de Lingu@gem, 17:e1721.
Finatto, M. J. B. (2024). Da linguagem simples à acessibilidade textual e terminológica: um percurso com e na linguística. In SILVA, A. H. P.; LAGARES, X. C. M. M. o., editor, Linguagem simples: para quem? A comunicação cidadã em debate, pages 86–110. ABRALIN, Campinas.
Heineman, D., Dou, Y., Maddela, M., and Xu, W. (2023). Dancing between success and failure: Edit-level simplification evaluation using SALSA. In Bouamor, H., Pino, J., and Bali, K., editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3466–3495, Singapore. Association for Computational Linguistics.
Ivanov, V. and Solnyshkina, M. (2020). A method for assessment of text complexity based on knowledge graphs. In Proceedings of the Linguistic Forum 2020: Language and Artificial Intelligence, volume 2852 of CEUR Workshop Proceedings, Moscow, Russia. CEUR-WS.org.
Kim, T. (2022). Revisiting the practical effectiveness of constituency parse extraction from pre-trained language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5398–5408, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Kincaid, J. P., Fishburne, R. P., Rogers, R. L., and Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for army enlisted personnel. Technical Report Research Branch Report 8-75, Chief of Naval Technical Training, Millington, TN, USA.
Leal, S., Duran, M., and Scarton, C. (2024). Nilc-metrix: assessing the complexity of written and spoken language in brazilian portuguese. Lang Resources & Evaluation, 58.
Leal, S. E., Duran, M. S., and Aluísio, S. M. (2018). A Nontrivial Sentence Corpus for the Task of Sentence Readability Assessment in Portuguese. In Bender, E. M., Derczynski, L., and Isabelle, P., editors, Proceedings of the 27th International Conference on Computational Linguistics, pages 401–413, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Leal, S. E., Serras, F. R., Finger, M., and Aluísio, S. M. (2026). Complexidade textual e suas tarefas relacionadas. In Caseli, H. M. and Nunes, M. G. V., editors, Processamento de Linguagem Natural: Conceitos, Técnicas e Aplicações em Português, volume 3, book chapter 6. BPLN, 4 edition.
Liu, Y., Fabbri, A., Chen, J., Zhao, Y., Han, S., Joty, S., Liu, P., Radev, D., Wu, C.-S., and Cohan, A. (2024). Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization. In Duh, K., Gomez, H., and Bethard, S., editors, Findings of the Association for Computational Linguistics: NAACL 2024, pages 4481–4501, Mexico City, Mexico. Association for Computational Linguistics.
Lopes, L. and Pardo, T. (2024). Towards portparser - a highly accurate parsing system for Brazilian Portuguese following the Universal Dependencies framework. In Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H. G., and Amaro, R., editors, Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 401–410, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Oliveira, L., Claro, D. B., and Souza, M. (2022). Dptoie: a portuguese open information extraction based on dependency analysis. Artif. Intell. Rev., 56(7):7015–7046.
Queiroz, B., Cavalcante, R., and Claro, D. (2023). Desafios da tarefa de extração de informação aberta: uma abordagem metodológica de um corpus automatizado até o corpus manual. In Anais do XIV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 388–392, Porto Alegre, RS, Brasil. SBC.
Saggion, H. and Hirst, G. (2017). Automatic text simplification, volume 32. Springer.
Scalercio, A., Bertotto, E., Jesus, S., Finatto, M. J., and Paes, A. (2026). Annotation guidelines and challenges for automatic simplification of Portuguese drug leaflets. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 2, pages 121–127, Salvador, Brazil. Association for Computational Linguistics.
Scalercio, A., Finatto, M., and Paes, A. (2024). Enhancing sentence simplification in Portuguese: Leveraging paraphrases, context, and linguistic features. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics ACL 2024, pages 15076–15091, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Scalercio, A. M. R. D. A., Souza, E. A. D., Finatto, M. J. B., and Paes, A. (2025). Evaluating LLMs for Portuguese sentence simplification with linguistic insights. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T., editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24452–24477, Vienna, Austria. Association for Computational Linguistics.
Shardlow, M., Alva-Manchego, F., Batista-Navarro, R., Bott, S., Calderon Ramirez, S., Cardon, R., François, T., Hayakawa, A., Horbach, A., Hülsing, A., Ide, Y., Imperial, J. M., Nohejl, A., North, K., Occhipinti, L., Rojas, N. P., Raihan, N., Ranasinghe, T., Salazar, M. S., Štajner, S., Zampieri, M., and Saggion, H. (2024). The BEA 2024 shared task on the multilingual lexical simplification pipeline. In Kochmar, E., Bexte, M., Burstein, J., Horbach, A., Laarmann-Quante, R., Tack, A., Yaneva, V., and Yuan, Z., editors, Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 571–589, Mexico City, Mexico. Association for Computational Linguistics.
Silva, S., Athaydes, A., Mane, B., Claro, D., Souza, M., Neto, F. M., Dantas, L., and Cavalcante, R. (2025). Desafios dos grafos de conhecimento: uma proposta de avaliação de sistemas openie. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 718–722, Porto Alegre, RS, Brasil. SBC.
Vásquez-Rodríguez, L., Shardlow, M., Przybyła, P., and Ananiadou, S. (2021). Investigating text simplification evaluation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 876–882, Online. Association for Computational Linguistics.
Wu, X. and Arase, Y. (2026). An in-depth evaluation of large language models in sentence simplification with error-based human assessment. ACM Trans. Intell. Syst. Technol., 17(4).
Xu, W., Napoles, C., Pavlick, E., Chen, Q., and Callison-Burch, C. (2016). Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415.
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2020). BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations (ICLR).
Al-Thanyyan, S. S. and Azmi, A. M. (2021). Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1–36.
Alva-Manchego, F., Scarton, C., and Specia, L. (2020). Data-driven sentence simplification: Survey and benchmark. Computational Linguistics, 46(1):135–187.
Alva-Manchego, F., Scarton, C., and Specia, L. (2021). The (un)suitability of automatic evaluation metrics for text simplification. Computational Linguistics, 47(4):861–889.
Alves, A., Miranda, P., Mello, R., and Nascimento, A. (2023). Automatic simplification of legal texts in portuguese using machine learning. In Legal Knowledge and Information Systems, pages 281–286. IOS Press.
Baez, A. and Saggion, H. (2023). LSLlama: Fine-tuned LLaMA for lexical simplification. In Štajner, S., Saggio, H., Shardlow, M., and Alva-Manchego, F., editors, Proceedings of the Second Workshop on Text Simplification, Accessibility and Readability, pages 102–108, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
Chandrasekar, R., Doran, C., and Bangalore, S. (1996a). Motivations and methods for text simplification. In COLING 1996 volume 2: The 16th international conference on computational linguistics.
Chandrasekar, R., Doran, C., and Srinivas, B. (1996b). Motivations and methods for text simplification. In COLING 1996 Volume 2: The 16th International Conference on Computational Linguistics.
de Menezes, L. C., Paes, A., and Finatto, M. J. B. (2023). Abordagem baseada em aumento de dados para avaliação automática de leiturabilidade. Domínios de Lingu@gem, 17:e1721.
Finatto, M. J. B. (2024). Da linguagem simples à acessibilidade textual e terminológica: um percurso com e na linguística. In SILVA, A. H. P.; LAGARES, X. C. M. M. o., editor, Linguagem simples: para quem? A comunicação cidadã em debate, pages 86–110. ABRALIN, Campinas.
Heineman, D., Dou, Y., Maddela, M., and Xu, W. (2023). Dancing between success and failure: Edit-level simplification evaluation using SALSA. In Bouamor, H., Pino, J., and Bali, K., editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3466–3495, Singapore. Association for Computational Linguistics.
Ivanov, V. and Solnyshkina, M. (2020). A method for assessment of text complexity based on knowledge graphs. In Proceedings of the Linguistic Forum 2020: Language and Artificial Intelligence, volume 2852 of CEUR Workshop Proceedings, Moscow, Russia. CEUR-WS.org.
Kim, T. (2022). Revisiting the practical effectiveness of constituency parse extraction from pre-trained language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5398–5408, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Kincaid, J. P., Fishburne, R. P., Rogers, R. L., and Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for army enlisted personnel. Technical Report Research Branch Report 8-75, Chief of Naval Technical Training, Millington, TN, USA.
Leal, S., Duran, M., and Scarton, C. (2024). Nilc-metrix: assessing the complexity of written and spoken language in brazilian portuguese. Lang Resources & Evaluation, 58.
Leal, S. E., Duran, M. S., and Aluísio, S. M. (2018). A Nontrivial Sentence Corpus for the Task of Sentence Readability Assessment in Portuguese. In Bender, E. M., Derczynski, L., and Isabelle, P., editors, Proceedings of the 27th International Conference on Computational Linguistics, pages 401–413, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Leal, S. E., Serras, F. R., Finger, M., and Aluísio, S. M. (2026). Complexidade textual e suas tarefas relacionadas. In Caseli, H. M. and Nunes, M. G. V., editors, Processamento de Linguagem Natural: Conceitos, Técnicas e Aplicações em Português, volume 3, book chapter 6. BPLN, 4 edition.
Liu, Y., Fabbri, A., Chen, J., Zhao, Y., Han, S., Joty, S., Liu, P., Radev, D., Wu, C.-S., and Cohan, A. (2024). Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization. In Duh, K., Gomez, H., and Bethard, S., editors, Findings of the Association for Computational Linguistics: NAACL 2024, pages 4481–4501, Mexico City, Mexico. Association for Computational Linguistics.
Lopes, L. and Pardo, T. (2024). Towards portparser - a highly accurate parsing system for Brazilian Portuguese following the Universal Dependencies framework. In Gamallo, P., Claro, D., Teixeira, A., Real, L., Garcia, M., Oliveira, H. G., and Amaro, R., editors, Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 401–410, Santiago de Compostela, Galicia/Spain. Association for Computational Lingustics.
Oliveira, L., Claro, D. B., and Souza, M. (2022). Dptoie: a portuguese open information extraction based on dependency analysis. Artif. Intell. Rev., 56(7):7015–7046.
Queiroz, B., Cavalcante, R., and Claro, D. (2023). Desafios da tarefa de extração de informação aberta: uma abordagem metodológica de um corpus automatizado até o corpus manual. In Anais do XIV Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 388–392, Porto Alegre, RS, Brasil. SBC.
Saggion, H. and Hirst, G. (2017). Automatic text simplification, volume 32. Springer.
Scalercio, A., Bertotto, E., Jesus, S., Finatto, M. J., and Paes, A. (2026). Annotation guidelines and challenges for automatic simplification of Portuguese drug leaflets. In Souza, M., de Dios-Flores, I., Santos, D., Freitas, L., Souza, J. W. d. C., and Ribeiro, E., editors, Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 2, pages 121–127, Salvador, Brazil. Association for Computational Linguistics.
Scalercio, A., Finatto, M., and Paes, A. (2024). Enhancing sentence simplification in Portuguese: Leveraging paraphrases, context, and linguistic features. In Ku, L.-W., Martins, A., and Srikumar, V., editors, Findings of the Association for Computational Linguistics ACL 2024, pages 15076–15091, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Scalercio, A. M. R. D. A., Souza, E. A. D., Finatto, M. J. B., and Paes, A. (2025). Evaluating LLMs for Portuguese sentence simplification with linguistic insights. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T., editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24452–24477, Vienna, Austria. Association for Computational Linguistics.
Shardlow, M., Alva-Manchego, F., Batista-Navarro, R., Bott, S., Calderon Ramirez, S., Cardon, R., François, T., Hayakawa, A., Horbach, A., Hülsing, A., Ide, Y., Imperial, J. M., Nohejl, A., North, K., Occhipinti, L., Rojas, N. P., Raihan, N., Ranasinghe, T., Salazar, M. S., Štajner, S., Zampieri, M., and Saggion, H. (2024). The BEA 2024 shared task on the multilingual lexical simplification pipeline. In Kochmar, E., Bexte, M., Burstein, J., Horbach, A., Laarmann-Quante, R., Tack, A., Yaneva, V., and Yuan, Z., editors, Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 571–589, Mexico City, Mexico. Association for Computational Linguistics.
Silva, S., Athaydes, A., Mane, B., Claro, D., Souza, M., Neto, F. M., Dantas, L., and Cavalcante, R. (2025). Desafios dos grafos de conhecimento: uma proposta de avaliação de sistemas openie. In Anais do XVI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pages 718–722, Porto Alegre, RS, Brasil. SBC.
Vásquez-Rodríguez, L., Shardlow, M., Przybyła, P., and Ananiadou, S. (2021). Investigating text simplification evaluation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 876–882, Online. Association for Computational Linguistics.
Wu, X. and Arase, Y. (2026). An in-depth evaluation of large language models in sentence simplification with error-based human assessment. ACM Trans. Intell. Syst. Technol., 17(4).
Xu, W., Napoles, C., Pavlick, E., Chen, Q., and Callison-Burch, C. (2016). Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415.
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2020). BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations (ICLR).
Publicado
19/10/2026
Como Citar
SILVA, Samuel Rios da; DANTAS, Larrissa; CLARO, Daniela Barreiro; PAES, Aline; FINATTO, Maria José; CAVALCANTE, Rerisson; RIBEIRO, Silvana.
Validating Semantic Content in Sentence Simplification using OpenIE and Relational Graphs. In: SIMPÓSIO BRASILEIRO DE TECNOLOGIA DA INFORMAÇÃO E DA LINGUAGEM HUMANA (STIL), 17. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 405-419.
DOI: https://doi.org/10.5753/stil.2026.26598.
