SoundEyes: Obstacle Audiodescription for People with Visual Impairment

• Jerson V. P. Gomes PUC Minas
• Wallace F. Oliveira PUC Minas
• Fellipe G. Oliveira PUC Minas
• Rafael H. N. Diniz PUC Minas
• Matheus A. Souza PUC Minas
• Felipe D. Cunha PUC Minas

DOI: https://doi.org/10.5753/sbcup.2025.8130

Abstract

Visually impaired individuals often face challenges when navigating unfamiliar or dynamic environments, where access to real-time spatial information is limited. This paper presents the development of SoundEyes, an assistive technology for visually impaired individuals that employs computer vision techniques to recognize objects and generate real-time audio descriptions. Designed for mobile devices with limited resources, the system uses an edge computing architecture and Bluetooth communication to ensure low latency and high autonomy. In practical tests with mobile devices, SoundEyes achieved a total response time of less than one second in HVGA mode on mid-range devices and demonstrated greater detection accuracy in XGA mode, showing promise for both dynamic and static environments.

References

AbdElminaam, D. S., Ahmed, I. A.-E., and Sakr, F. (2022). SCBIoT: Smart cane for blinds using IoT. In International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pages 371–377.

Devi, S. K. and Subalalitha, C. N. (2021). Deep learning based audio assistive system for visually impaired people. Computers, Materials and Continua, 71(1):1205–1219.

Dissanayake, D. M. L. V., Rajapaksha, R. G. M. D. R. P., Prabhashawara, U. P., Solanga, S. A. D. S., and Anuradha Jayakody, J. A. D. C. (2021). Guide-me: Voice authenticated indoor user guidance system. In IEEE Ubiquitous Computing, Electronics & Mobile Comm. Conf. (UEMCON), pages 0509–0514.

Google LLC (2024). Google Text-to-Speech (gTTS) API.

Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications.

Jocher, G., Chaurasia, A., and Qiu, J. (2023). Ultralytics yolov8.

Osama, M., Yehia, A., Mohamed, S., Sherief, R., Elmasry, N., Adel, V., and Hamdy, A. (2021). Design and implementation of visually impaired assistant system. In Int. Mobile, Intelligent, and Ubiquitous Comp. Conf., pages 303–310.

Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only look once: Unified, real-time object detection.

Supekar, A. and Patil, S. (2022). Design and development of portable navigation system for disabled person using image, text and audio. In IEEE Delhi Section Conference (DELCON), pages 1–4.

World Health Organization (2019). World Report on Vision. World Health Organization, Geneva, Switzerland. Acesso em: 19 jan. 2025.