Pictogram Prediction in Alternative Communication Boards: a Mapping Study
Resumo
Alternative communication boards (ACB) are tools that try to compensate for the difficulties faced by people with complex communication needs. Generally, these tools consist of a mobile application in which the user can construct sentences by arranging pictograms (picture+label pair representing a concept) in sequence. This study systematically maps the literature on pictogram prediction in ACBs. We analyzed eight studies to investigate how computational methods are used for pictogram prediction, how these proposals are evaluated, and the studies' outcomes regarding user communication improvement. The main findings indicate the usage of different methods for pictogram prediction and a mixture of automatic and expert evaluation, which lead to inconclusive outcomes regarding user communication improvement.
Palavras-chave:
Augmentative and alternative communication, pictogram predicion, mapping study
Referências
Ascari, R. E. d. O. S., Pereira, R., and Silva, L. (2018). Mobile interaction for augmentative and alternative communication: a systematic mapping. Journal on Interactive Systems, 9(2).
ASHA (2022). Augmentative and alternative communication. [link]. June, 2022.
Aydin, O. and Diken, I. H. (2020). Studies comparing augmentative and alternative communication systems (aac) applications for individuals with autism spectrum disorder: A systematic review and meta-analysis. Education and training in autism and developmental disabilities, 55(2):119–141.
Berenguer, C., Martínez, E. R., De Stasio, S., and Baixauli, I. (2022). Parentsrsquo; perceptions and experiences with their childrenrsquo;s use of augmentative/alternative communication: A systematic review and qualitative meta-synthesis. International Journal of Environmental Research and Public Health, 19(13).
Beukelman, D. R. and Light, J. C. (2013). Augmentative & Alternative Communication: Supporting Children and Adults with Complex Communication Needs. Paul H. Brookes Baltimore.
Chen, L., Babar, M. A., and Zhang, H. (2010). Towards an evidence-based understanding of electronic data sources. In 14th International conference on evaluation and assessment in software engineering (EASE), pages 1–4.
Dada, S., van der Walt, C., May, A. A., and Murray, J. (2022). Intelligent assistive technology devices for persons with dementia: A scoping review. Assistive Technology, 0(0):1–14. PMID: 34644248.
Donato, C., Spencer, E., and Arthur-Kelly, M. (2018). A critical synthesis of barriers and facilitators to the use of aac by children with autism spectrum disorder and their communication partners. Augmentative and Alternative Communication, 34(3):242–253.
Dudy, S. and Bedrick, S. (2018). Compositional Language Modeling for Icon-Based Augmentative and Alternative Communication. Proceedings of the conference. Association for Computational Linguistics. Meeting, 2018:25–32.
Fabbri, S., Silva, C., Hernandes, E., Octaviano, F., Di Thommazo, A., and Belgamo, A. (2016). Improvements in the start tool to better support the systematic review process. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, EASE ’16, New York, NY, USA. Association for Computing Machinery.
Franco, N., Silva, E., Lima, R., and Fidalgo, R. (2018). Towards a reference architecture for augmentative and alternative communication systems. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 29, page 1073.
Garcia, L. F., de Oliveira, L. C., and de Matos, D. M. (2016). Evaluating pictogram prediction in a location-aware augmentative and alternative communication system. Assistive Technology, 28(2):83–92.
García, P., Lleida, E., Castán, D., Marcos, J. M., and Romero, D. (2015). Context-Aware Communicator for All. In Antona, M. and Stephanidis, C., editors, Universal Access in Human-Computer Interaction. Access to Today’s Technologies, pages 426–437, Cham. Springer International Publishing.
Goldberg, Y. and Hirst, G. (2017). Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers
Hervás, R., Bautista, S., Méndez, G., Galván, P., and Gervás, P. (2020). Predictive composition of pictogram messages for users with autism. Journal of Ambient Intelligence and Humanized Computing, 11(11):5649–5664.
Jurafsky, D. and Martin, J. (2019). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. 3 edition.
Kitchenham, B. (2004). Procedures for Performing Systematic Reviews.
Light, J. and McNaughton, D. (2012). The changing face of augmentative and alternative communication: Past, present, and future challenges. Augmentative and Alternative Communication, 28(4):197–204. PMID: 23256853.
MacWhinney, B. (2014). The CHILDES project: Tools for analyzing talk, Volume II: The database. Psychology Press.
Martínez-Santiago, F., Díaz-Galiano, M. C., Ureña-López, L. A., and Mitkov, R. (2015). A semantic grammar for beginning communicators. Knowledge-Based Systems, 86:158–172.
PEREIRA, Jayr; FRANCO, Natália; FIDALGO, Robson. A semantic grammar for augmentative and alternative communication systems. In: International Conference on Text, Speech, and Dialogue. Springer, Cham, 2020. p. 257-264.
Pereira, J., Pena, C., de Melo, M., Cartaxo, B., Soares, S., and Fidalgo, R. (2019). Facilitators and barriers to using alternative and augmentative communication systems by aphasic: Therapists perceptions. In 2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS), pages 349–354
Pereira, J. A., Macêdo, D., Zanchettin, C., de Oliveira, A. L. I., and do Nascimento Fidalgo, R. (2022). Pictobert: Transformers for next pictogram prediction. Expert Systems with Applications, 202:117231.
Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic mapping studies in software engineering. In 12th International Conference on Evaluation and Assessment in Software Engineering (EASE) 12, pages 1–10.
Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.
Saturno, C. E., Ramirez, A. R. G., Conte, M. J., Farhat, M., and Piucco, E. C. (2015). An augmentative and alternative communication tool for children and adolescents with cerebral palsy. Behaviour & Information Technology, 34(6):632–64
Scarlini, B., Pasini, T., and Navigli, R. (2020). With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages ”3528 – 3539”. Association for Computational Linguistics.
Schwab, D., Trial, P., Vaschalde, C., Vial, L., Esperança-Rodier, E., and Lecouteux, B. (2020). Providing semantic knowledge to a set of pictograms for people with disabilities: a set of links between wordnet and arasaac: Arasaac-wn. In LREC, pages 166–171.
Sennott, S. C., Akagi, L., Lee, M., and Rhodes, A. (2019). AAC and Artificial Intelligence (AI). Topics in language disorders, 39(4):389–403.
ASHA (2022). Augmentative and alternative communication. [link]. June, 2022.
Aydin, O. and Diken, I. H. (2020). Studies comparing augmentative and alternative communication systems (aac) applications for individuals with autism spectrum disorder: A systematic review and meta-analysis. Education and training in autism and developmental disabilities, 55(2):119–141.
Berenguer, C., Martínez, E. R., De Stasio, S., and Baixauli, I. (2022). Parentsrsquo; perceptions and experiences with their childrenrsquo;s use of augmentative/alternative communication: A systematic review and qualitative meta-synthesis. International Journal of Environmental Research and Public Health, 19(13).
Beukelman, D. R. and Light, J. C. (2013). Augmentative & Alternative Communication: Supporting Children and Adults with Complex Communication Needs. Paul H. Brookes Baltimore.
Chen, L., Babar, M. A., and Zhang, H. (2010). Towards an evidence-based understanding of electronic data sources. In 14th International conference on evaluation and assessment in software engineering (EASE), pages 1–4.
Dada, S., van der Walt, C., May, A. A., and Murray, J. (2022). Intelligent assistive technology devices for persons with dementia: A scoping review. Assistive Technology, 0(0):1–14. PMID: 34644248.
Donato, C., Spencer, E., and Arthur-Kelly, M. (2018). A critical synthesis of barriers and facilitators to the use of aac by children with autism spectrum disorder and their communication partners. Augmentative and Alternative Communication, 34(3):242–253.
Dudy, S. and Bedrick, S. (2018). Compositional Language Modeling for Icon-Based Augmentative and Alternative Communication. Proceedings of the conference. Association for Computational Linguistics. Meeting, 2018:25–32.
Fabbri, S., Silva, C., Hernandes, E., Octaviano, F., Di Thommazo, A., and Belgamo, A. (2016). Improvements in the start tool to better support the systematic review process. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, EASE ’16, New York, NY, USA. Association for Computing Machinery.
Franco, N., Silva, E., Lima, R., and Fidalgo, R. (2018). Towards a reference architecture for augmentative and alternative communication systems. In Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), volume 29, page 1073.
Garcia, L. F., de Oliveira, L. C., and de Matos, D. M. (2016). Evaluating pictogram prediction in a location-aware augmentative and alternative communication system. Assistive Technology, 28(2):83–92.
García, P., Lleida, E., Castán, D., Marcos, J. M., and Romero, D. (2015). Context-Aware Communicator for All. In Antona, M. and Stephanidis, C., editors, Universal Access in Human-Computer Interaction. Access to Today’s Technologies, pages 426–437, Cham. Springer International Publishing.
Goldberg, Y. and Hirst, G. (2017). Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers
Hervás, R., Bautista, S., Méndez, G., Galván, P., and Gervás, P. (2020). Predictive composition of pictogram messages for users with autism. Journal of Ambient Intelligence and Humanized Computing, 11(11):5649–5664.
Jurafsky, D. and Martin, J. (2019). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. 3 edition.
Kitchenham, B. (2004). Procedures for Performing Systematic Reviews.
Light, J. and McNaughton, D. (2012). The changing face of augmentative and alternative communication: Past, present, and future challenges. Augmentative and Alternative Communication, 28(4):197–204. PMID: 23256853.
MacWhinney, B. (2014). The CHILDES project: Tools for analyzing talk, Volume II: The database. Psychology Press.
Martínez-Santiago, F., Díaz-Galiano, M. C., Ureña-López, L. A., and Mitkov, R. (2015). A semantic grammar for beginning communicators. Knowledge-Based Systems, 86:158–172.
PEREIRA, Jayr; FRANCO, Natália; FIDALGO, Robson. A semantic grammar for augmentative and alternative communication systems. In: International Conference on Text, Speech, and Dialogue. Springer, Cham, 2020. p. 257-264.
Pereira, J., Pena, C., de Melo, M., Cartaxo, B., Soares, S., and Fidalgo, R. (2019). Facilitators and barriers to using alternative and augmentative communication systems by aphasic: Therapists perceptions. In 2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS), pages 349–354
Pereira, J. A., Macêdo, D., Zanchettin, C., de Oliveira, A. L. I., and do Nascimento Fidalgo, R. (2022). Pictobert: Transformers for next pictogram prediction. Expert Systems with Applications, 202:117231.
Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic mapping studies in software engineering. In 12th International Conference on Evaluation and Assessment in Software Engineering (EASE) 12, pages 1–10.
Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.
Saturno, C. E., Ramirez, A. R. G., Conte, M. J., Farhat, M., and Piucco, E. C. (2015). An augmentative and alternative communication tool for children and adolescents with cerebral palsy. Behaviour & Information Technology, 34(6):632–64
Scarlini, B., Pasini, T., and Navigli, R. (2020). With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages ”3528 – 3539”. Association for Computational Linguistics.
Schwab, D., Trial, P., Vaschalde, C., Vial, L., Esperança-Rodier, E., and Lecouteux, B. (2020). Providing semantic knowledge to a set of pictograms for people with disabilities: a set of links between wordnet and arasaac: Arasaac-wn. In LREC, pages 166–171.
Sennott, S. C., Akagi, L., Lee, M., and Rhodes, A. (2019). AAC and Artificial Intelligence (AI). Topics in language disorders, 39(4):389–403.
Publicado
16/11/2022
Como Citar
PEREIRA, Jayr A.; MEDEIROS, Sheyla de; ZANCHETTIN, Cleber; FIDALGO, Robson do N..
Pictogram Prediction in Alternative Communication Boards: a Mapping Study. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 33. , 2022, Manaus.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 705-717.
DOI: https://doi.org/10.5753/sbie.2022.225217.
