What if HMDs could generate audio descriptions of real scenes for visually impaired people? - Mauricio, Claudio; Domingues, Gustavo; Padua, Ivã; Peres, Fabiana; Teixeira, João Marcelo (Anais do Simpósio de Realidade Virtual e Aumentada (SVR), 2024)
This paper explores the feasibility and effectiveness of using the Meta Quest 3 head-mounted display to generate real-time audio descriptions of real scenes for visually impaired individuals. The objective is to enhance the spatial awareness and interaction capabilities of visually impaired users by providing continuous, detailed audio descriptions of their surroundings. The methodology involves leveraging the Meta Quest 3 to capture real-time visual data, which is then processed and converted into audio descriptions using AI-driven software. Experiments were conducted in various real-world environments to test the accuracy and usability of the audio description system. Key findings indicate that the Meta Quest 3 can effectively deliver real-time audio descriptions with high accuracy, significantly improving the navigation and situational awareness of visually impaired users. This study demonstrates the potential of HMD technology to create more inclusive and accessible experiences, highlighting the importance of continued innovation in assistive technologies for the visually impaired​.