Contextual exploration of mathematical formulae on the web for people with visual disabilities in brazil with an open-source screen reader


Despite the technological advances in screen readers, specific contents such as mathematics are still a problem for people with visual disabilities. Due to the complexity of mathematical notations and their abstract nature, mathematics is a science that presents challenges for understanding by means other than visual. Thus, reading mathematical content using screen readers currently faces problems such as ambiguity, help to understand long formulae, and identifying specific elements. Besides, even with reading problems, few screen readers support reading this type of content in Portuguese. This study presents the implementation and evaluation of an add-on for reading and exploring mathematical formulae with NVDA considering contextual in-formula exploration and navigation. This way, the add-on enables the possibility of internally exploring the formulae using contextual delineations in mathematical elements with support to the Brazilian variant of the Portuguese language. The add-on, entitled Access8Math-NavMatBR, was based on the open-source Access8Math add-on. The version of the add-on was developed and evaluated in usability tests with the participation of six people with visual disabilities. Tests with the new system demonstrated an improvement in the understanding of formulae and an improvement in access to specific elements with exploration with an abstraction of formulae. The evaluation identified 52 instances of issues identified by users, involving problems with the commands and interaction approaches to explore the formulae and their parts, the verbalization by the screen reader and the platform's structure. The paper presents design implications for systems for reading mathematical formulae in the Brazilian context and considerations on the exploration patterns used by Brazilian users when reading and browsing mathematical formulae.

Palavras-chave: Accessibility, Visual Disability, Mathematics, Interactive digital material


Stênio Abreu, Jorge Sassaki Resende Silva, Graciela Pires dos Anjos, Hérlon Manollo Cândido Guedes, Soraia Silva Prietch, Paula Christina Figueira Cardoso, and André Pimenta Freire. 2019. Usability Evaluation of a Resource to Read Mathematical Formulae in a Screen Reader for People with Visual Disabilities. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (Vitória, Espírito Santo, Brazil) (IHC '19). Association for Computing Machinery, New York, NY, USA, Article 53, 11 pages.

Dragan Ahmetovic, Tiziana Armano, Cristian Bernareggi, Anna Capietto, Sandro Coriasco, Boris Doubrov, Alexandr Kozlovskiy, and Nadir Murru. 2020. Automatic Tagging of Formulae in PDF Documents and Assistive Technologies for Visually Impaired People: The LaTeX Package axessibility 3.0. In ICCHP - Future Perspectives of AT, eAccessibility and eInclusion. Springer, online, 69--73.

Agnieszka Bier and Zdzisław Sroczyński. 2015. Adaptive Math-To-Speech Interface. In Proceedings of the Mulitimedia, Interaction, Design and Innnovation (Warsaw, Poland) (MIDI '15). Association for Computing Machinery, New York, NY, USA, Article 7, 9 pages.

Nattapat Boonprakong, Patcharida Pudpadee, Thanarat H Chalidabhongse, and Proadpran Punyabukkana. 2017. Reading mathematical expression in Thai. In Proceedings of the 11th International Convention on Rehabilitation Engineering and Assistive Technology. Singapore Therapeutic, Assistive & Rehabilitative Technologies (START) Centre, Singapore, 9.

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.

Jolanta Brzostek-Pawłowska, Małgorzata Rubin, and Andrzej Salamończyk. 2019. Enhancement of math content accessibility in EPUB3 educational publications. New Review of Hypermedia and Multimedia 25, 1--2 (2019), 31--56. arXiv:

Davide Cervone, Peter Krautzberger, and Volker Sorge. 2016. Employing semantic analysis for enhanced accessibility features in MathJax. In 2016 13th IEEE Annual Consumer Communications & Networking Conference (CCNC). IEEE, Las Vegas, NV, USA, 1129--1134.

Luiz Felipe da Paixão Silva, Otavio de Faria Oliveira, Evelise Roman Corbalan Góis Freire, Rosana Maria Mendes, and André Pimenta Freire. 2017. How Much Effort is Necessary for Blind Users to Read Web-based Mathematical Formulae?: A Comparison Using Task Models with Different Screen Readers. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems (Joinville, Brazil) (IHC 2017). ACM, New York, NY, USA, Article 29, 10 pages.

Luiz Felipe da Paixão Silva, Antônio Armando de O Barbosa, Evelise Roman Corbalan Góis Freire, Paula Christina Figueira Cardoso, Rafael Serapilha Durelli, and André Pimenta Freire. 2018. Content-based Navigation Within Mathematical Formulae On the Web For Blind Users and Its Impact On Expected User Effort. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion. ACM, New York, NY, 23--32.

Mirella Alves de Lima, Daniela Rodrigues, Patrícia Vasconcelos Almeida, Paula Christina Figueira Cardoso, and André Pimenta Freire. 2019. Análise de verbalizações de fórmulas matemáticas por professores com experiência no ensino de pessoas com deficiência visual/Analysis of mathematical formulas verbalizations by teachers with experience in teaching visually impaired people. Revista de Estudos da Linguagem 27, 3 (2019), 1371--1397.

Iyad Abu Doush and Enrico Pontelli. 2009. Building a programmable architecture for non-visual navigation of mathematics: Using rules for guiding presentation and switching between modalities. In International Conference on Universal Access in Human-Computer Interaction. San Diego, California, USA, 3--13.

Alistair DN Edwards, Heather McCartney, and Flavio Fogarolo. 2006. Lambda:: a multimodal approach to making mathematics accessible to blind students. In Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility. ACM, Portland, Oregon, USA, 48--54.

Alistair D. N. Edwards. 1989. Modelling blind users interactions with an auditory computer interface. International Journal of Man-Machine Studies 30, 5 (1989), 575--589.

Islam Elkabani and Rached Zantout. 2015. A framework for helping the visually impaired learn and practice math. In 2015 5th International Conference on Information & Communication Technology and Accessibility (ICTA). IEEE, Marrakech, Morocco, 1--5.

Everis Brasil. 2018. Resultados da Pesquisa do Uso de Leitores de Tela. Research Results. Everis Brasil, São Paulo, SP. Disponível em [link], último acesso em 12/01/2021.

S Fajardo-Flores and D Archambault. 2014. Evaluation of a Prototype of a Multimodal Interface to Work with Mathematics. AMSE IFRATH 75 (2014), 106--118.

Helder Ferreira and Diamantino Freitas. 2005. AudioMath: using MathML for speaking mathematics. In XATA05 - XML and Associated Technologies. Universidade do Minho, Braga, Portugal.

Lois Frankel, Beth Brownstein, and Neil Soiffer. 2014. Navigable, customizable TTS for algebra. In 28th Annual International Technology and Persons with Disabilities Conference Scientific/Research Proceedings. California State University, Northridge, San Diego, CA, 13--24.

Pavel Gaura. 2002. REMathEx --- Reader and Editor of the Mathematical Expressions for Blind Students. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, and Wolfgang Zagler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 486--493.

Martin Gruber, Jindrich Matousek, Zdenek Hanzlícek, Zdenek Krnoul, and Zbynek Zajíc. 2016. ARET-Automatic Reading of Educational Texts for Visually Impaired Students. In INTERSPEECH - Annual Conference of the International Speech Communication Association. ISCA, San Francisco, CA, 383--384.

Ann P Gulley, Luke A Smith, Jordan A Price, Logan C Prickett, and Matthew F Ragland. 2017. Process-driven math: An auditory method of mathematics instruction and assessment for students who are blind or have low vision. Journal of visual impairment & blindness 111, 5 (2017), 465--471.

Albino Szesz Junior. 2021. Math2Text: Ferramenta Tecnológica para Acessibilidade de Estudantes Cegos a Expressões Matemáticas. Ph.D. Dissertation. Programa de Pós-Graduação em Ensino de Ciência e Tecnologia, Universidade Tecnológica Federal do Paraná, Ponta Grossa, PR.

Albino Szesz Junior, Lucas Ribeiro Mendes, and Sani de Carvalho Rutz da Silva. 2020. Math2Text: Software para geração e conversão de equações matemáticas em texto-limitações e possibilidades de inclusão. Revista Ibérica de Sistemas e Tecnologias de Informação 37 (2020), 99--115.

Andrew Kirkpatrick, Joshue O Connor, Alastair Campbell, and Michael Cooper. 2018. Web Content Accessibility Guidelines (WCAG) 2.1.

Michał Maćkowski, Piotr Brzoza, Marek Zabka, and Dominik Spinczyk. 2018. Multimedia platform for mathematics' interactive learning accessible to blind people. Multimedia Tools and Applications 77, 5 (2018), 6191--6208.

Jindřich Matoušek, Zdeněk Hanzlíček, Michal Campr, Zdeněk Krňoul, Pavel Campr, and Martin Grber. 2011. Web-based system for automatic reading of technical documents for vision impaired students. In International Conference on Text, Speech and Dialogue. Springer, Berlin, 364--371.

Paúl Mejía, Luiz César Martini, Felipe Grijalva, Julio César Larco, and Juan Carlos Rodríguez. 2021. A survey on mathematical software tools for visually impaired persons: A practical perspective. IEEE Access 9 (2021), 66929--66947.

Robert R Miner, Patrick D F Ion, and David Carlisle. 2014. Mathematical Markup Language (MathML) Version 3.0 2nd Edition. W3C Recommendation. W3C, Cambridge, MA. Disponível em, último acesso em 12/01/2021.

Christopher Power, Helen Petrie, David Swallow, Emma Murphy, Bláithín Gallagher, and Carlos A Velasco. 2013. Navigating, discovering and exploring the web: strategies used by people with print disabilities on interactive websites. In IFIP Conference on Human-Computer Interaction. Springer, Berlin, 667--684.

M Fayezur Rahman. 2005. Hierarchical Manipulation of Mathematical Expressions for Visually Impaired Students. In Proceedings of the 5th Winona Computer Science Undergraduate Research Symposium. Winona State University, Winona, MN, 8--15.

T.V. Raman. 1994. Audio system for technical readings. Ph.D. Dissertation. Cornell University, Dept. of Computer Science., Ithaca, NY, USA.

Andrzej Salamonczyk and Jolanta Brzostek-Pawlowska. 2015. Translation of MathML formulas to Polish text, example applications in teaching the blind. In 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF). IEEE, Gdynia, Poland, 240--244.

Neil Soiffer. 2005. MathPlayer: Web-based Math Accessibility. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, MD, USA) (Assets '05). ACM, New York, NY, USA, 204--205.

Neil Soifer. 2007. MathPlayer V2.1: Web-based Math Accessibility. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Tempe, Arizona, USA) (Assets '07). ACM, New York, NY, USA, 257--258.

Neil Soiffer. 2015. Browser-independent Accessible Math. In Proceedings of the 12th Web for All Conference (Florence, Italy) (W4A '15). ACM, New York, NY, USA, Article 28, 3 pages.

Volker Sorge, Charles Chen, T. V. Raman, and David Tseng. 2014. Towards Making Mathematics a First Class Citizen in General Screen Readers. In Proceedings of the 11th Web for All Conference (Seoul, Korea) (W4A '14). ACM, New York, NY, USA, Article 40, 10 pages.

Robert D Stevens, Alistair DN Edwards, and Philip A Harling. 1997. Access to mathematics for visually disabled students through multimodal interaction. Human-computer interaction 12, 1--2 (1997), 47--92.

Norman B. Sutherland. 1972. Braille Display Device. Patente número US3659354A.

Masakazu Suzuki, Toshihiro Kanahori, Nobuyuki Ohtake, and Katsuhito Yamaguchi. 2004. An integrated OCR software for mathematical documents and its output with accessibility. In International Conference on Computers for Handicapped Persons. Springer, 648--655.

Woody Tseng. 2018. Access8Math. Disponível em [link], último acesso em 4 de junho de 2022.

Takayuki Watanabe. 2007. Experimental Evaluation of Usability and Accessibility of Heading Elements. In Proceedings of the 2007 International Cross-disciplinary Conference on Web Accessibility (W4A) (Banff, Canada) (W4A '07). ACM, New York, NY, USA, 157--164.

Takayuki Watanabe. 2009. Experimental evaluation of usability and accessibility of heading elements. Disability and Rehabilitation: Assistive Technology 4, 4 (2009), 236--247.

Katsuhito Yamaguchi, Masakazu Suzuki, and Toshihiro Kanahori. 2014. Braille Capability in Accessible e-Textbooks for Math and Science. In Computers Helping People with Special Needs, Klaus Miesenberger, Deborah Fels, Dominique Archambault, Petr Peňáz, and Wolfgang Zagler (Eds.). Springer International Publishing, Cham, 557--563.
GUEDES, Hérlon Manollo Cândido; CARDOSO, Paula C. F.; WATANABE, Willian Massami; FREIRE, André Pimenta. Contextual exploration of mathematical formulae on the web for people with visual disabilities in brazil with an open-source screen reader. In: SIMPÓSIO BRASILEIRO SOBRE FATORES HUMANOS EM SISTEMAS COMPUTACIONAIS (IHC), 21. , 2022, Diamantina. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2022 .