Handwritten Text Entry in Virtual Reality Using Gesture Recognition and Word Prediction - Silva, João; Carvalho, Kaique; Cardoso, Luciana; Félix, Juliana; Soares, Fabrizzio; Nascimento, Thamer Horbylon (Anais Estendidos do Simpósio de Realidade Virtual e Aumentada (SVR); 2025: Anais Estendidos do XXVII Simpósio de Realidade Virtual e AumentadaDO - 10.5753/svr_estendido.2025.15771, 2025)
This work presents an approach for text entry in virtual reality (VR) environments, using handwritten letters drawn in the air as a form of natural interaction. The system was developed for the Meta Quest 2 device and is composed of different integrated modules: real-time capture of gestures performed by the user, character recognition using a convolutional neural network (CNN) trained with the EMNIST dataset, and the construction of words through a Trie structure, which enables efficient term search based on recognized letters. Furthermore, the final selection of words is performed based on their usage frequency, which allows prioritizing more probable terms within a common linguistic context. The method allows for complete word input in VR, with consistent performance in the identification of individual letters and automatic suggestion generation, demonstrating that it can provide a fluid, intuitive experience compatible with the immersive interaction proposal in three-dimensional environments.