Accessible AR Framework for Interactive Narratives with Guided Phases

• Paulo Lasalvia UFPE / CESAR
• Geraldo Cruz UFPE
• Francisco Simões UFPE
• Robson Fidalgo UFPE

Resumo

Augmented Reality (AR) technologies have been increasingly applied in educational and therapeutic contexts, yet they still face important accessibility challenges for users with cognitive impairments. This paper introduces a proposal for an accessible AR framework based on interactive narratives structured in guided phases and supported by multimodal feedback (visual, auditory, and tactile). The goal is to promote digital inclusion and autonomy for users with cognitive disabilities by delivering step-by-step instructions enhanced with narration and other sensory cues. We developed a modular architecture in Unity, integrating plane detection through ARFoundation and distinct modules for phases, audio, and feedback. The system allows therapists to customize activity sequences (narratives) tailored to individual user profiles. We discuss potential use cases in education and rehabilitation, aligning the framework with accessibility guidelines (WCAG) applied to AR. Furthermore, we compare our proposal to related works, highlighting differentiators such as integrated narration, touch-based interaction, modular phase design, and cognitive accessibility focus. Finally, expected results and an initial technical validation with specialists are described, along with future directions for broader applicability.

Palavras-chave: augmented reality, accessibility, interactive narratives, digital inclusion, AR-assisted therapy

Referências

R. S. Baragash, H. Al-Samarraie, A. I. Alzahrani, and O. Alfarraj. Augmented reality in special education: a meta-analysis of single-subject design studies. European Journal of Special Needs Education, 35(2), 286–300, 2020.

D. F. Cihak, E. J. Moore, R. E. Wright, M. M. Gibbons, and C. M. Smith. Evaluating augmented reality to complete a chain task for elementary students with autism. Journal of Special Education Technology, 31(3), 177–187, 2016.

W3C. Web content accessibility guidelines (WCAG) 2.1. [link], 2018. Accessed June 2025.

World Wide Web Consortium (W3C) Web Accessibility Initiative. XR accessibility user requirements. W3C Working Group Note, 2021. [link].

D. Wolf, D. Besserer, K. Sejunaite, A. Schuler, M. Riepe, and E. Rukzio. A demonstration of care: an augmented reality support system for geriatric inpatients with mild cognitive impairment. Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (MUM), Pisa, Italy, 1–4, 2019.

Instituto Brasileiro de Geografia e Estatistica (IBGE). Desemprego e informalidade sao maiores entre as pessoas com deficiencia. Agencia de Noticias IBGE, September 2022.

M. Akcayir and G. Akcayir. Advantages and challenges associated with augmented reality for education: a systematic review. Educational Research Review, 20, 1–11, 2017.

A. C. dos Santos and S. B. L. Ferreira. Accessibility in augmented reality educational applications: a systematic review. Anais do Simposio Brasileiro de Informatica na Educacao (SBIE), SBC, 476–485, 2020.

Y. Zhao, J. Kim, S. Coppola, C. Stier, and J. E. Gilbert. Designing inclusive augmented reality applications for individuals with autism spectrum disorder. Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, 341–343, 2019.

F. Garzotto and D. Fornaroli. Bridging the accessibility gap in mobile augmented reality for children with disabilities. Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, 17–24, 2014.

A. Dengel, M. Z. Iqbal, S. Grafe, and E. Mangina. A review on augmented reality authoring toolkits for education. Frontiers in Virtual Reality, 3, 798032, 2022.

R. O. Kellems, G. Cacciatore, and B. Osborne. Using augmented reality to teach job skills to individuals with intellectual and developmental disabilities. Journal of Special Education Technology, 35(1), 3–14, 2020.

T. Alves, A. P. Chaves, and A. Matsunaga. A framework for the development of accessible and customizable educational applications using augmented reality. Anais do Workshop de Informatica na Escola (WIE), SBC, 206–215, 2021.

A. Di Serio, M. B. Ibanez, and C. D. Kloos. Impact of an augmented reality system on students motivation for a visual art course. Computers & Education, 68, 586–596, 2013.

M. F. Story. Maximizing usability: The principles of universal design. Assistive Technology, 10(1), 4–12, 1998.

J. Freitas, A. P. Chaves, R. Souto, and A. Matsunaga. Visual programming interfaces for non-technical users in inclusive education. Anais do Simposio Brasileiro de Informatica na Educacao (SBIE), SBC, 250–259, 2020.

C. Moro, Z. Phelps, M. Redmond, and A. Stromberga. Multimodal learning in health sciences using augmented reality: a systematic review. Journal of Educational Technology Systems, 46(3), 385–412, 2017.

B. Coelho, M. Barros, and D. Lopes. Design de interacoes multimodais para apoio a aprendizagem de pessoas com deficiencia cognitiva. Anais do Congresso Brasileiro de Informatica na Educacao (CBIE), 1183–1192, 2022.

Y.-J. Chang, Y.-S. Kang, and P.-C. Huang. An augmented reality (AR)-based vocational task prompting system for people with cognitive impairments. Research in Developmental Disabilities, 34(10), 3049–3056, 2013.

A. Oliveira, P. Costa, and F. Lima. Immersive narratives in augmented reality: Enhancing user engagement. Proceedings of the 24th Symposium on Virtual and Augmented Reality (SVR), IEEE, 12–19, 2022.

J. Silva and M. Santos. Exploring multimodal feedback in augmented reality for enhanced learning. Proceedings of the 23rd Symposium on Virtual and Augmented Reality (SVR), IEEE, 75–82, 2021.

M. Chen, A. Monroy-Hernandez, and M. Sra. SceneAR: Scene-based micro narratives for sharing and remixing in augmented reality. IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 294–303, 2021.