What if AI Could Revolutionize Literature Reviews in Virtual Reality and Mental Health? - Guedes, Paulo; Abrantes, Pedro; Teixeira, João Marcelo; Teichrieb, Veronica (Anais do Congresso Latino-Americano de Software Livre e Tecnologias Abertas (Latinoware); 2024: Anais do XXI Congresso Latino-Americano de Software Livre e Tecnologias AbertasDO - 10.5753/latinoware.2024.245673, 2024)
The rapid advancement of Large Language Models (LLMs) has opened new avenues for automating and enhancing the process of systematic literature reviews. This study investigates the effectiveness of three state-of-the-art LLMs — ChatGPT4o, LLaMA, and Gemini — in conducting literature reviews on the intersection of virtual reality and mental health, focusing on works by a renowned author in the field, Mel Slater. We defined two objective and two subjective questions to evaluate the performance of these models against a ground truth dataset. Our findings reveal significant insights into the accuracy, reliability, and limitations of each model, providing a comprehensive analysis of their potential and constraints. This study aims to guide future applications of LLMs in academic research, highlighting the transformative potential of these technologies in conducting systematic