The Synthesized Speech of Mathematical Expressions: A Study to Improve the Cognitive Process of the Visually Impaired Student

• Adriana Souza Federal Institute of Bahia / University of Porto
• Diamantino Freitas University of Porto

DOI: https://doi.org/10.5753/sbie.2022.225721

Abstract

The synthesized speech of mathematical content still presents challenges for students who use screen readers, among which we can mention inappropriate pauses and long auditory exits, making it difficult to memorize this type of content. In this study we carried out two experiments with the objective of identifying and analyzing processes capable of reducing the cognitive overload of speech synthesized from mathematical expressions. The first experiment aimed to verify the difficulties encountered by students with visual impairments, in addition to analyzing a proposed pause model. The second experiment sought to understand the cognitive processes for memorization of mathematical expressions, through the eye tracking technique of sighted people. Although some results were not conclusive, the research proved to be relevant as it points out directions that can minimize mental load and consequently improve the student's cognitive process in reading mathematical expressions rendered in MathML.

Keywords: Synthesized Speech, Mathematical Expressions, Cognitive Process, Visual Impairment

References

Andrá, Chiara, Paulina Lindström, Ferdinando Arzarello, Kenneth Holmqvist, Ornella Robutti, e Cristina Sabena. 2015. “Reading Mathematics Representations: An Eye - Tracking Study”. International Journal of Science and Mathematics Education 13:237–59.

Bansal, Akashdeep e M. Balakrishnan. 2021. “Evaluating Cognitive Complexity of Algebraic Equations”. The Journal on Technology and Persons with Disabilities 170.

Bates, Enda e Donal Fitzpatrick. 2010. “Spoken Mathematics using Prosody, Earcons and Spearcons 2 Visual and Audible Representations of Mathematical Material”.

Bitter, Marc. 2013. “Braille in Mathematics Education”. Radboud University Nijmegen Braille.

Boersma, Paul e David Weenink. 2018. “Praat: doing phonetics by computer”.

Campione, Estelle e Jean Véronis. 2002. “A Large-Scale Multilingual Study of Silent Pause Duration”. Speech Prosody 2002, abril, 4.

Carlisle, David, Patrick Ion, e Robert Miner. 2014. Mathematical markup language (MathML) version 3.0 2nd edition, W3C recommendation, W3C.

Chomsky, Noam. 1964. “Formal discussion of the development of grammar in child language by W. Miller & S. Ervin”. The Acquisition of language.

Chomsky, Noam. 2009. Syntactic structures. De Gruyter Mouton. Ernest, Paul. 1987. “a Model of the Cognitive Meaning of Mathematical Expressions”. British Journal of Educational Psychology 57(3):343–70.

Ferreira, Helder e Diamantino Freitas. 2004. “Enhancing the accessibility of mathematics for blind people: The audiomath project”. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 3118:678–85.

Frankel, Lois e Beth Brownstein. 2016. “An Evaluation of the Usefulness of Prosodic and Lexical Cues for Understanding Synthesized Speech of Mathematics”. ETS Research Report Series 2016(2):1–19.

Frankel, Lois, Beth Brownstein, Neil Soiffer, e Eric Hansen. 2016. “Development and Initial Evaluation of the ClearSpeak Style for Automated Speaking of Algebra”. ETS Research Report Series 2016(2):1–43.

Hartmann, Matthias. 2015. “Numbers in the eye of the beholder: What do eye movements reveal about numerical cognition?” Cognitive Processing 16:245–48.

Kintsch, Walter. 1978. “Comprehension and memory of text”. Handbook of learning and cognitive processes 6:57–86.

Kohlhase, Andrea e Michael Kohlhase. 2021. “Conceptual Design for an Eye-Tracking Experiment on Formula Linebreaking”. in 14th Conference on Intelligent Computer Mathematics (CICM 2021).

Maier, H. R., Z. Kapelan, J. Kasprzyk, J. Kollat, L. S. Matott, M. C. Cunha, G. C. Dandy, M. S. Gibbs, E. Keedwell, A. Marchi, A. Ostfeld, D. Savic, D. P. Solomatine, J. A. Vrugt, A. C. Zecchin, B. S. Minsker, E. J. Barbour, G. Kuczera, F. Pasha, A. Castelletti, M. Giuliani, e P. M. Reed. 2014. “Evolutionary algorithms and other metaheuristics in water resources: Current status, research challenges and future directions”. Environmental Modelling & Software 62:271–99.

Nazemi, Azadeh, Iain Murray, e Nazanin Mohammadi. 2012. “Mathspeak: An Audio Method for Presenting Mathematical Formulae to Blind Students”. P. 48--52 in 5th International Conference on Human System Interactions.

O’Malley, Michael H., Dean R. Kloker, e Benay Dara-Abrams. 1973. “Recovering Parentheses From Spoken Algebraic Expressions”. IEEE Transactions on Audio and Electroacoustics 21(3):217–20.

Plummer, Patrick, Melissa DeWolf, Miriam Bassok, Peter C. Gordon, e Keith J. Holyoak. 2017. “Reasoning strategies with rational numbers revealed by eye tracking”. Attention, Perception, and Psychophysics 79(5):1426–37.

Rayner, Keith. 1998. “Eye movements in reading and information processing: 20 years of research”. Psychological Bulletin 124:372.

Reich, S. S. 1980. “Significance of pauses for speech perception”. Journal of Psycholinguist 379–389.

Röder, Brigitte, Frank Rösler, e Helen J. Neville. 2001. “Auditory memory in congenitally blind adults: a behavioral-electrophysiological investigation”. Cognitive Brain Research 11(2):289–303.

Rumelhart, David e Donald A. Norman. 1981. “Analogical processes in learning”. Cognitive skills and their acquisition 335–59.

Rumsey, Deborah J. 2016. “How to interpret a correlation coefficient r”. Statistics For Dummies.

Schindler, Maike, · Achim, e J. Lilienthal. 2022. “Students’ collaborative creative process and its phases in mathematics: an explorative study using dual eye tracking and stimulated recall interviews”. 54:163–78.

Souza, Adriana e Diamantino Freitas. 2018. “Tecnologias Assistivas para Apoiar o Ensino e Aprendizagem de Pessoas com Deficiência Visual na Matemática: Uma Revisão Sistemática da Literatura”. P. 923 in Anais do XXIX Simpósio Brasileiro de Informática na Educação (SBIE 2018). Vol. 1. Brazilian Computer Society (Sociedade Brasileira de Computação - SBC).

Stevens, Robert David. 1996. “Principles for the design of auditory interfaces to present complex information to blind people”. University of York.

Zaiontz, Charles. 2022. “Real statistics using Excel. 2015”. in [Computer software] http://www.realstatistics.com/free-download/ (accessed 2 May 2022).