LLM-Generated RDF Triples for Agricultural Species: A Comparative Evaluation Using AGROVOC Grounding

• Leonardo Vianna do Nascimento IFRS https://orcid.org/0000-0002-2197-3106

DOI: https://doi.org/10.5753/encompif.2026.21874

Resumo

The construction of RDF knowledge bases for specialized domains is costly and requires close collaboration between domain experts and knowledge engineers. This paper evaluates three commercial LLMs — Claude, ChatGPT, and Gemini — in generating RDF Turtle triples for 38 plant species relevant to Brazilian agriculture. All models received an identical prompt combining five prompt engineering techniques, including few-shot exemplification and external file grounding via a CSV file with correct AGROVOC URIs. Outputs were assessed for AGROVOC URI precision, common name correctness against Embrapa reference sources, and syntactic conformance. Claude achieved perfect URI precision (100%) and the highest recall for common names (82.9%). ChatGPT reached the highest common name precision (96.1%) but poor URI precision (16.2%). Gemini showed similar recall (59.8%) and worse URI precision (8.1%). All models produced syntactically valid Turtle. Results indicate that grounding effectiveness varies across models and that programmatic URI validation is essential in LLM-assisted knowledge base construction.

Referências

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Caracciolo, C., Stellato, A., Morshed, A., Johannsen, G., Rajbhandari, S., Jaques, Y., and Keizer, J. (2013). The agrovoc linked dataset. Semantic Web, 4(3):341–348.

Caufield, J. H., Hegde, H., Emonet, V., Harris, N. L., Joachimiak, M. P., Matentzoglu, N., Kim, H., Moxon, S., Reese, J. T., Haendel, M. A., et al. (2024). Structured prompt interrogation and recursive extraction of semantics (spires): A method for populating knowledge bases using zero-shot learning. Bioinformatics, 40(3):btae104.

Drury, B., Fernandes, R., Moura, M.-F., and de Andrade Lopes, A. (2019). A survey of semantic web technology for agriculture. Information Processing in Agriculture, 6(4):487–501.

Frey, J., Meyer, L.-P., Arndt, N., Brei, F., and Bulert, K. (2023). Benchmarking the abilities of large language models for rdf knowledge graph creation and comprehension: how well do llms speak turtle? In ISWC: Workshop Deep Learning for Knowledge Graphs.

Gong, R. and Li, X. (2025). The application progress and research trends of knowledge graphs and large language models in agriculture. Computers and electronics in agriculture, 235:110396.

Mavridis, A., Tegos, S., Anastasiou, C., Papoutsoglou, M., and Meditskos, G. (2025). Large language models for intelligent rdf knowledge graph construction: results from medical ontology mapping. Frontiers in Artificial Intelligence, 8:1546179.

Norouzi, S. S., Barua, A., Christou, A., Gautam, N., Eells, A., Hitzler, P., and Shimizu, C. (2025). Ontology population using llms. In Handbook on Neurosymbolic AI and Knowledge Graphs, pages 421–438. IOS Press.

Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., and Wu, X. (2024). Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7):3580–3599.

Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., Li, Y., Gupta, A., Han, H., Schulhoff, S., et al. (2024). The prompt report: A systematic survey of prompt engineering techniques. arXiv preprint arXiv:2406.06608.