LínguaQuiz: Integration of Image Recognition and LLM in a Game for Speech Therapy

• Pedro Henrique Brandão Oliveira Bastos UEFS
• Victor Travassos Sarinho UEFS

DOI: https://doi.org/10.5753/sbgames_estendido.2025.9891

Abstract

Introduction: The use of digital games as a complementary tool in therapeutic processes has shown promise, especially in the field of speech therapy. Objective: This work presents the development of the game LínguaQuiz, designed to assist in the rehabilitation of oral movements through image capture and analysis using a Large Language Model (LLM). Methodology or Steps: The game uses the device’s camera to capture images of the user’s tongue movements, which are processed by a visual LLM. Depending on the tongue movements performed by the player, actions are interpreted and corresponding commands are sent to a quiz-style game, allowing therapy sessions to be conducted in a gamified format. Results: It is expected that LínguaQuiz will help make speech therapy rehabilitation exercises more dynamic, motivating, and effective, expanding the possibilities of support for clinical work.

Keywords: Digital game, Image recognition, Speech therapy rehabilitation, Visual LLM, Gametherapy

References

Schmid, M. (2009). Reinforcing motor re-training and rehabilitation through games: a machine-learning perspective. Frontiers in neuroengineering, 2:1016.

Sousa, A. d. S. et al. (2011). Serious games para a fonoaudiologia: uma abordagem voltada à terapia em motricidade orofacial.

Weiss, P. L., Rand, D., Katz, N., e Kizony, R. (2004). Video capture virtual reality as a flexible and effective rehabilitation tool. Journal of neuroengineering and rehabilitation, 1:1–12.