Scaling and Adapting Large Language Models for Portuguese Open Information Extraction: A Comparative Study of Fine-Tuning and LoRA

• Alan Melo UFBA
• Bruno Cabral UFBA
• Daniela Barreiro Claro UFBA

Resumo

This paper comprehensively investigates the efficacy of different adaptation techniques for Large Language Models (LLMs) in the context of Open Information Extraction (OpenIE) for Portuguese. We compare Full Fine-Tuning (FFT) and Low-Rank Adaptation (LoRA) across a model with 0.5B parameters. Our study evaluates the impact of model size and adaptation method on OpenIE performance, considering precision, recall, and F1 scores, as well as computational efficiency during training and inference phases. We contribute to a high-performing LLM and novel insights into the trade-offs between model scale, adaptation technique, and cross-lingual transferability in the OpenIE task. Our findings reveal significant performance variations across different configurations, with LoRA demonstrating competitive results. We also analyze the linguistic nuances in the Portuguese OpenIE that pose challenges for models primarily trained on English data. This research advances our understanding of LLM adaptation for specialized NLP tasks and provides practical guidelines for deploying these models in resource-constrained and multilingual scenarios. Our work has implications for the broader cross-lingual open information extraction field and contributes to the ongoing discourse on efficient fine-tuning strategies for large pre-trained models.