Automatic audio classification of pseudoword readings for large-scale assessment of reading fluency in children in the literacy phase

• Elias Cyrino de Assis Federal University of Juiz de Fora
• André Luiz Vasconcelos Ferreira Federal University of Juiz de Fora
• Cristiano Nascimento Silva Federal University of Juiz de Fora
• Jairo Francisco de Souza Federal University of Juiz de Fora https://orcid.org/0000-0002-0911-7980

DOI: https://doi.org/10.5753/sbie.2022.224701

Abstract

The pseudoword reading test is used in several large-scale assessments that seek to assess the reading fluency of children in the literacy phase. As with the other items that make up the fluency assessments, assessing the pseudoword test is a costly task, from which the development of ASR systems to automate the assessment process arises. In this work, an approach to automatically evaluate pseudoword readings using a pre-trained self-supervised model is presented. Three experiments were carried out with different strategies to calculate reading metrics. The performance of the strategies was compared with the evaluation of the human corrector.

Keywords: ASR, large-scale fluency assessment, pseudoword reading

References

Almeida Silva, W., Carchedi, L., Gomes Jr, J., Souza, J., Barrere, E., and Souza, J. (2021). A framework for large-scale automatic fluency assessment. International Journal of Distance Education Technologies, 19.

Baevski, A., Zhou, H., Mohamed, A., and Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. In 10.48550/ARXIV.2006.11477. arXiv.

Batista, A. A. G. (2011). Alfabetização, leitura e ensino de português: desafios e perspectivas curriculares. Revista Contemporânea de Educação, 6(12):246–272.

Ceccon, D. L. and Porto, J. B. (2020). Bcs: Jogos digitais no auxílio do desenvolvimento de crianças especiais com atraso na linguagem. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 522–531. SBC.

Cresswell, J., Schwantner, U., and Waters, C. (2015). A Review of International Large-Scale Assessments in Education. PISA, The World Bank, Washington, D.C./OECD Publishing, Paris.

de Mira Gobbo, M. R., Barbosa, C. R., Morandini, M., and Mafort, F. (2019). Aplicativo para ganho de vocabulário e auxílio na alfabetização destinado às crianças com transtorno do espectro autista. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 30, page 1111.

Dias, N. M., Léon, C. B. R., Pazeto, T. d. C. B., Martins, G. L. L., Pereira, A. P. P., and Seabra, A. G. (2016). Avaliação da leitura no brasil: Revisão da literatura no recorte 2009-2013. Psicologia: teoria e prática, 18(1):113–128.

Duchateau, J., Cleuren, L., Van hamme, H., and Ghesquière, P. (2007). Automatic assessment of children’s reading level. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 1:1210–1213.

Elenius, D. and Blomberg, M. (2004). Comparing speech recognition for adults and children. Proceedings of FONETIK 2004, pages 156–159.

Farrall, M. L. and Ashby, J. (2019). The role of assessment in structured literacy. Perspectives on Language and Literacy, 45(3):31–35.

Hautala, J., Heikkilä, R., Nieminen, L., Rantanen, V., Latvala, J.-M., and Richardson, U. (2020). Identification of reading difficulties by a digital game-based assessment technology. Journal of Educational Computing Research, 58(5):1003–1028.

Junior, A. C., Casanova, E., Soares, A., de Oliveira, F. S., Oliveira, L., Junior, R. C. F., da Silva, D. P. P., Fayet, F. G., Carlotto, B. B., Gris, L. R. S., and Aluísio, S. M. (2021). Coraa: a large corpus of spontaneous and prepared speech manually validated for speech recognition in brazilian portuguese. arXiv.

Mechelli, A., Gorno-Tempini, M. L., and Price, C. J. (2003). Neuroimaging studies of word and pseudoword reading: consistencies, inconsistencies, and limitations. Journal of cognitive neuroscience, 15(2):260–271.

National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. National Institute of Child Health and Human Development, National . . . .

Pantoja, J., Sousa, A., and de Araújo Júnior, R. M. (2018). Alfa autista: uma aplicação mobile para o auxílio na alfabetização do autista através de método fônico. um etudo de caso na apae-marabá. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 29, page 1873.

Passos, C., Fernandes, I., and Goldschmidt, R. (2019). Elaboração e avaliação de projeto de aprendizagem apoiado em jogos educacionais digitais: Um relato de experiência com alunos em alfabetização. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 30, page 674.

Pinheiro, Â. M. V. and de Araújo Vilhena, D. (2022). Teste de reconhecimento de palavras e pseudopalavras: validades de conteúdo e externa. Signo, 47(88):145–161.

Proença, J., Lopes, C., Tjalve, M., Stolcke, A., Candeias, S., and Perdigão, F. (2017a). Automatic Evaluation of Children Reading Aloud on Sentences and Pseudowords. In Proc. Interspeech 2017, pages 2749–2753.

Proença, J., Lopes, C., Tjalve, M., Stolcke, A., Candeias, S., and Perdigão, F. (2017b). Detection of Mispronunciations and Disfluencies in Children Reading Aloud. In Proc. Interspeech 2017, pages 1437–1441.

Proença, J. D. L. (2018). Automatic assessment of reading ability of children. PhD thesis, Faculdade de Ciências e Tecnologia da Universidade de Coimbra.

Rasinski, T. V. (2004). Assessing reading fluency. Pacific Resources for Education and Learning (PREL).

Salles, J. F. d. and Parente, M. A. d. M. P. (2007). Avaliação da leitura e escrita de palavras em crianças de 2ª série: abordagem neuropsicológica cognitiva. Psicologia: Reflexão e Crítica, 20(2):220–228.

Shivakumar, P. G. and Georgiou, P. (2020). Transfer learning from adult to children for speech recognition: Evaluation, analysis and recommendations. Computer speech & language, 63:101077.

Silva, W. A., Gomes Jr, J., Knop, I., Barrére, E., and Souza, J. (2019). Talk2me: Uma abordagem computacional para auxiliar na identificação de falhas no processo de alfabetização. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 30, page 723.

Vaessen, N. and van Leeuwen, D. A. (2021). Fine-tuning wav2vec2 for speaker recognition. arXiv preprint arXiv:2109.15053.

Wagemaker, H. (2014). International large-scale assessments: From research to policy. Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis, pages 11–36.

Yılmaz, E., Pelemans, J., and Van hamme, H. (2014). Automatic assessment of children’s reading with the flavor decoding using a phone confusion model. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH.