What if AI Could Revolutionize Literature Reviews in Virtual Reality and Mental Health? - Guedes, Paulo; Abrantes, Pedro; Teixeira, João Marcelo; Teichrieb, Veronica (Anais do Congresso Latino-Americano de Software Livre e Tecnologias Abertas (Latinoware), 2024)
The rapid advancement of Large Language Models (LLMs) has opened new avenues for automating and enhancing the process of systematic literature reviews. This study investigates the effectiveness of three state-of-the-art LLMs — ChatGPT4o, LLaMA, and Gemini — in conducting literature reviews on the intersection of virtual reality and mental health, focusing on works by a renowned author in the field, Mel Slater. We defined two objective and two subjective questions to evaluate the performance of these models against a ground truth dataset. Our findings reveal significant insights into the accuracy, reliability, and limitations of each model, providing a comprehensive analysis of their potential and constraints. This study aims to guide future applications of LLMs in academic research, highlighting the transformative potential of these technologies in conducting systematic