VisioAux: a crowdsource approach for accessibility evaluation in mobile devices
Resumo
Introduction: The rapid growth in mobile device usage has increased the need for accessible applications. Automated accessibility testing approaches were developed to support accessibility evaluation, but they have limitations such as low coverage of features and screens and the lack of prioritization of issues by real-world impact. Objective: This paper introduces an automated crowdsourcing-based strategy to increase coverage and provide impact-based prioritization of accessibility issues in mobile applications. Method: We developed VisioAux, an accessibility testing service integrated into mobile apps and deployed on end users’ devices. As users navigate apps in their daily activities, VisioAux performs runtime accessibility checks on apps under test. VisioAux collects reports from multiple devices and consolidates results to estimate coverage and issue severity for a given app. Results: A preliminary evaluation compared VisioAux with Google Accessibility Scanner, assessed its performance impact, and verified its crowdsourcing capabilities. Results indicate comparable detection performance, negligible performance overhead, and successful aggregation of accessibility reports from diverse devices.
Palavras-chave:
Accessibility, Testing, Monitoring, Mobile, Android, Crowdsourcing
Referências
Aashutosh, K. (2023). Mobile Accessibility Rituals: WCAG 2.1 Made Easy for Mobile Apps. Amazon Italia.
Alonso, F., Fuertes, J. L., González, A. L., e Martínez, L. (2010). On the testability of wcag 2.0 for beginners. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A), W4A ’10, New York, NY, USA. Association for Computing Machinery.
Alotaibi, A. S., Chiou, P. T., e Halfond, W. G. (2021). Automated repair of size-based inaccessibility issues in mobile applications. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 730–742. IEEE.
Alotaibi, A. S., Chiou, P. T., e Halfond, W. G. (2022). Automated detection of talkback interactive accessibility failures in android applications. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), pages 232–243. IEEE.
Alotaibi, A. S., Chiou, P. T., Tawsif, F. M., e Halfond, W. G. (2023). Scalefix: An automated repair of ui scaling accessibility issues in android applications. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 147–159. IEEE.
Alshayban, A., Ahmed, I., e Malek, S. (2020). Accessibility issues in android apps: state of affairs, sentiments, and ways forward. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 1323–1334.
Alshayban, A. e Malek, S. (2022). Accessitext: automated detection of text accessibility issues in android apps. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 984–995.
Alyahya, S. (2020). Crowdsourced software testing: A systematic literature review. Information and Software Technology, 127:106363.
Android Developers (2024a). Accessibility checks with espresso. [link]. Accessed: 2025-04-09.
Android Developers (2024b). Accessibility service - manifest declaration. [link]. Accessed: 2025-08-04.
Android Developers (2024c). Accessibilityservice. [link]. Accessed: 2025-04-14.
Android Developers (2024d). Add build dependencies. [link]. Accessed: 2025-08-05.
Android Developers (2024e). App not responding (ANR). [link]. Accessed: 2025-05-10.
Android Developers (2024f). Permissions overview. [link]. Accessed: 2025-08-04.
Android Developers (2024g). Processes and Threads. [link]. Accessed: 2025-05-10.
Apple Inc. (2024). UIView | Threading Considerations. [link] Threading-considerations. Accessed: 2025-05-10.
BBC (n.d.). Accessibility - mobile accessibility guidelines. [link]. Accessed: 2025-04-05.
Bi, T., Xia, X., Lo, D., Grundy, J., Zimmermann, T., e Ford, D. (2022). Accessibility in software practice: A practitioner’s perspective. ACM Transactions on Software Engineering and Methodology.
Cerwall, P. (2021). Ericsson mobility report. [link]. Accessed: 2025-08-19.
Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., e Liu, Y. (2021a). Accessible or not an empirical investigation of android app accessibility. IEEE Transactions on Software Engineering.
Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., e Liu, Y. (2021b). Accessible or not? an empirical investigation of android app accessibility. IEEE Transactions on Software Engineering, 48(10):3954–3968.
Da Costa Nunes, E. H., Castro, K. V. G. d., Anjos, E. L. d. F. d., Silva, P. V. d. S., e Monteiro, I. T. (2024). Color contrast compliance: Investigating constrast requirements in brazilian websites. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.
Dias, J., Carvalho, D., Paredes, H., Martins, P., Rocha, T., e Barroso, J. (2021). Automated evaluation tools for web and mobile accessibility: A systematic literature review. In International Conference on Innovations in Bio-Inspired Computing and Applications, pages 447–456. Springer.
Eler, M. M., Rojas, J. M., Ge, Y., e Fraser, G. (2018a). Automated accessibility testing of mobile apps. In 2018 IEEE 11th International Conference on Software Testing, Verifcation and Validation (ICST), pages 116–126. IEEE.
Eler, M. M., Rojas, J. M., Ge, Y., e Fraser, G. (2018b). Automated accessibility testing of mobile apps. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST), pages 116–126. IEEE.
Firebase (2025). Firebase Crashlytics Documentation. [link]. Accessed: 2025-04-11.
Fok, R., Zhong, M., Ross, A. S., Fogarty, J., e Wobbrock, J. O. (2022). A large-scale longitudinal analysis of missing label accessibility failures in android apps. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–16.
Giebel, J. (2020). Accessibility evaluation tools for android mobile applications. Tagungsband, page 42.
Google (2024). Accessibility test framework for android. [link]. Accessed: 2025-04-14.
Google (2024a). Espresso. [link]. Accessed: 2025-04-11.
Google (2024b). Improve your code with lint checks. [link]. Accessed: 02-jun-2024.
Google (2024). Kotlin coroutines on android. [link]. Accessed: 2025-08-03.
Kochhar, P. S., Thung, F., Nagappan, N., Zimmermann, T., e Lo, D. (2015). Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pages 1–10. IEEE.
Lam, W., Wu, Z., Li, D., Wang, W., Zheng, H., Luo, H., Yan, P., Deng, Y., e Xie, T. (2017). Record and replay for android: Are we there yet in industrial cases? In Proceedings of the 2017 11th joint meeting on foundations of software engineering, pages 854–859.
Leite, M. V. R., Scatalon, L. P., Freire, A. P., e Eler, M. M. (2021). Accessibility in the mobile development industry in brazil: Awareness, knowledge, adoption, motivations and barriers. Journal of Systems and Software, 177:110942.
Lewthwaite, S., Horton, S., e Coverdale, A. (2023). Workplace approaches to teaching digital accessibility: establishing a common foundation of awareness and understanding. Frontiers in Computer Science, 5.
Martin, L., Baker, C., Shinohara, K., e Elglaly, Y. N. (2022). The landscape of accessibility skill set in the software industry positions. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility, pages 1–4.
Mateus, D. A., Silva, C. A., Eler, M. M., e Freire, A. P. (2020). Accessibility of mobile applications: evaluation by users with visual impairment and by automated tools. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, pages 1–10.
Matos, M., Seixas Pereira, L., e Duarte, C. (2023). Evaluation of the accessibility of mobile applications: Current approaches and challenges. In International Conference on Human-Computer Interaction, pages 352–371. Springer.
Mehralian, F., Salehnamadi, N., e Malek, S. (2021). Data-driven accessibility repair revisited: on the effectiveness of generating labels for icons in android apps. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 107–118.
Patel, R., Breton, P., Baker, C. M., El-Glaly, Y. N., e Shinohara, K. (2020). Why software is not accessible: Technology professionals’ perspectives and challenges. In Extended abstracts of the 2020 CHI conference on human factors in computing systems, pages 1–9.
Robolectric (2024). Robolectric: Unit testing framework for Android. [link]. Accessed: 2025-04-11.
Salehnamadi, N., Alshayban, A., Lin, J.-W., Ahmed, I., Branham, S., e Malek, S. (2021). Latte: Use-case and assistive-service driven automated accessibility testing framework for android. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–11.
Salehnamadi, N., He, Z., e Malek, S. (2023). Assistive-technology aided manual accessibility testing in mobile apps, powered by record-and-replay. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20.
Salehnamadi, N., Mehralian, F., e Malek, S. (2022). Groundhog: An automated accessibility crawler for mobile apps. in 2022 37th ieee. In ACM International Conference on Automated Software Engineering. IEEE, ACM New York, NY, USA, Rochester, Michigan, USA.
Silva, C., Eler, M. M., e Fraser, G. (2018a). A survey on the tool support for the automatic evaluation of mobile accessibility. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pages 286–293.
Silva, C., Eler, M. M., e Fraser, G. (2018b). A survey on the tool support for the automatic evaluation of mobile accessibility. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pages 286–293.
Song, S., Bu, J., Artmeier, A., Shi, K., Wang, Y., Yu, Z., e Wang, C. (2018). Crowdsourcing-based web accessibility evaluation with golden maximum likelihood inference. Proc. ACM Hum.-Comput. Interact., 2(CSCW).
Stol, K.-J. e Fitzgerald, B. (2014). Two’s company, three’s a crowd: a case study of crowdsourcing software development. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, page 187–198, New York, NY, USA. Association for Computing Machinery.
Tanenbaum, A. (2009). Modern operating systems. Pearson Education, Inc.,.
UXCam (2025). Leading mobile analytics tools. [link]. Accessed: 2025-08-08.
W3C (1999). Web content accessibility guidelines 2.2. [link]. Accessed: 2025-04-05.
(WAI), W. W. A. I. (2008). Understanding conformance requirements. [link]. Accessed: 2025-04-14.
Xu, Y., Li, Z., Liu, H., e Liu, Y. (2023). Agaa: An android gui accessibility adapter for low vision users. In 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), pages 412–421. IEEE.
Yan, S. e Ramachandran, P. (2019a). The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS), 12(1):1–31.
Yan, S. e Ramachandran, P. (2019b). The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS), 12(1):1–31.
Zhang, M., Liu, H., Chen, C., Gao, G., Li, H., e Zhao, J. (2023a). Accessfxer: Enhancing gui accessibility for low vision users with r-gcn model. IEEE Transactions on Software Engineering.
Zhang, Y., Chen, S., Fan, L., Chen, C., e Li, X. (2023b). Automated and context-aware repair of color-related accessibility issues for android apps. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1255–1267.
Alonso, F., Fuertes, J. L., González, A. L., e Martínez, L. (2010). On the testability of wcag 2.0 for beginners. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A), W4A ’10, New York, NY, USA. Association for Computing Machinery.
Alotaibi, A. S., Chiou, P. T., e Halfond, W. G. (2021). Automated repair of size-based inaccessibility issues in mobile applications. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 730–742. IEEE.
Alotaibi, A. S., Chiou, P. T., e Halfond, W. G. (2022). Automated detection of talkback interactive accessibility failures in android applications. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), pages 232–243. IEEE.
Alotaibi, A. S., Chiou, P. T., Tawsif, F. M., e Halfond, W. G. (2023). Scalefix: An automated repair of ui scaling accessibility issues in android applications. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 147–159. IEEE.
Alshayban, A., Ahmed, I., e Malek, S. (2020). Accessibility issues in android apps: state of affairs, sentiments, and ways forward. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 1323–1334.
Alshayban, A. e Malek, S. (2022). Accessitext: automated detection of text accessibility issues in android apps. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 984–995.
Alyahya, S. (2020). Crowdsourced software testing: A systematic literature review. Information and Software Technology, 127:106363.
Android Developers (2024a). Accessibility checks with espresso. [link]. Accessed: 2025-04-09.
Android Developers (2024b). Accessibility service - manifest declaration. [link]. Accessed: 2025-08-04.
Android Developers (2024c). Accessibilityservice. [link]. Accessed: 2025-04-14.
Android Developers (2024d). Add build dependencies. [link]. Accessed: 2025-08-05.
Android Developers (2024e). App not responding (ANR). [link]. Accessed: 2025-05-10.
Android Developers (2024f). Permissions overview. [link]. Accessed: 2025-08-04.
Android Developers (2024g). Processes and Threads. [link]. Accessed: 2025-05-10.
Apple Inc. (2024). UIView | Threading Considerations. [link] Threading-considerations. Accessed: 2025-05-10.
BBC (n.d.). Accessibility - mobile accessibility guidelines. [link]. Accessed: 2025-04-05.
Bi, T., Xia, X., Lo, D., Grundy, J., Zimmermann, T., e Ford, D. (2022). Accessibility in software practice: A practitioner’s perspective. ACM Transactions on Software Engineering and Methodology.
Cerwall, P. (2021). Ericsson mobility report. [link]. Accessed: 2025-08-19.
Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., e Liu, Y. (2021a). Accessible or not an empirical investigation of android app accessibility. IEEE Transactions on Software Engineering.
Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., e Liu, Y. (2021b). Accessible or not? an empirical investigation of android app accessibility. IEEE Transactions on Software Engineering, 48(10):3954–3968.
Da Costa Nunes, E. H., Castro, K. V. G. d., Anjos, E. L. d. F. d., Silva, P. V. d. S., e Monteiro, I. T. (2024). Color contrast compliance: Investigating constrast requirements in brazilian websites. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.
Dias, J., Carvalho, D., Paredes, H., Martins, P., Rocha, T., e Barroso, J. (2021). Automated evaluation tools for web and mobile accessibility: A systematic literature review. In International Conference on Innovations in Bio-Inspired Computing and Applications, pages 447–456. Springer.
Eler, M. M., Rojas, J. M., Ge, Y., e Fraser, G. (2018a). Automated accessibility testing of mobile apps. In 2018 IEEE 11th International Conference on Software Testing, Verifcation and Validation (ICST), pages 116–126. IEEE.
Eler, M. M., Rojas, J. M., Ge, Y., e Fraser, G. (2018b). Automated accessibility testing of mobile apps. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST), pages 116–126. IEEE.
Firebase (2025). Firebase Crashlytics Documentation. [link]. Accessed: 2025-04-11.
Fok, R., Zhong, M., Ross, A. S., Fogarty, J., e Wobbrock, J. O. (2022). A large-scale longitudinal analysis of missing label accessibility failures in android apps. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–16.
Giebel, J. (2020). Accessibility evaluation tools for android mobile applications. Tagungsband, page 42.
Google (2024). Accessibility test framework for android. [link]. Accessed: 2025-04-14.
Google (2024a). Espresso. [link]. Accessed: 2025-04-11.
Google (2024b). Improve your code with lint checks. [link]. Accessed: 02-jun-2024.
Google (2024). Kotlin coroutines on android. [link]. Accessed: 2025-08-03.
Kochhar, P. S., Thung, F., Nagappan, N., Zimmermann, T., e Lo, D. (2015). Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pages 1–10. IEEE.
Lam, W., Wu, Z., Li, D., Wang, W., Zheng, H., Luo, H., Yan, P., Deng, Y., e Xie, T. (2017). Record and replay for android: Are we there yet in industrial cases? In Proceedings of the 2017 11th joint meeting on foundations of software engineering, pages 854–859.
Leite, M. V. R., Scatalon, L. P., Freire, A. P., e Eler, M. M. (2021). Accessibility in the mobile development industry in brazil: Awareness, knowledge, adoption, motivations and barriers. Journal of Systems and Software, 177:110942.
Lewthwaite, S., Horton, S., e Coverdale, A. (2023). Workplace approaches to teaching digital accessibility: establishing a common foundation of awareness and understanding. Frontiers in Computer Science, 5.
Martin, L., Baker, C., Shinohara, K., e Elglaly, Y. N. (2022). The landscape of accessibility skill set in the software industry positions. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility, pages 1–4.
Mateus, D. A., Silva, C. A., Eler, M. M., e Freire, A. P. (2020). Accessibility of mobile applications: evaluation by users with visual impairment and by automated tools. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, pages 1–10.
Matos, M., Seixas Pereira, L., e Duarte, C. (2023). Evaluation of the accessibility of mobile applications: Current approaches and challenges. In International Conference on Human-Computer Interaction, pages 352–371. Springer.
Mehralian, F., Salehnamadi, N., e Malek, S. (2021). Data-driven accessibility repair revisited: on the effectiveness of generating labels for icons in android apps. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 107–118.
Patel, R., Breton, P., Baker, C. M., El-Glaly, Y. N., e Shinohara, K. (2020). Why software is not accessible: Technology professionals’ perspectives and challenges. In Extended abstracts of the 2020 CHI conference on human factors in computing systems, pages 1–9.
Robolectric (2024). Robolectric: Unit testing framework for Android. [link]. Accessed: 2025-04-11.
Salehnamadi, N., Alshayban, A., Lin, J.-W., Ahmed, I., Branham, S., e Malek, S. (2021). Latte: Use-case and assistive-service driven automated accessibility testing framework for android. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–11.
Salehnamadi, N., He, Z., e Malek, S. (2023). Assistive-technology aided manual accessibility testing in mobile apps, powered by record-and-replay. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20.
Salehnamadi, N., Mehralian, F., e Malek, S. (2022). Groundhog: An automated accessibility crawler for mobile apps. in 2022 37th ieee. In ACM International Conference on Automated Software Engineering. IEEE, ACM New York, NY, USA, Rochester, Michigan, USA.
Silva, C., Eler, M. M., e Fraser, G. (2018a). A survey on the tool support for the automatic evaluation of mobile accessibility. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pages 286–293.
Silva, C., Eler, M. M., e Fraser, G. (2018b). A survey on the tool support for the automatic evaluation of mobile accessibility. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, pages 286–293.
Song, S., Bu, J., Artmeier, A., Shi, K., Wang, Y., Yu, Z., e Wang, C. (2018). Crowdsourcing-based web accessibility evaluation with golden maximum likelihood inference. Proc. ACM Hum.-Comput. Interact., 2(CSCW).
Stol, K.-J. e Fitzgerald, B. (2014). Two’s company, three’s a crowd: a case study of crowdsourcing software development. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, page 187–198, New York, NY, USA. Association for Computing Machinery.
Tanenbaum, A. (2009). Modern operating systems. Pearson Education, Inc.,.
UXCam (2025). Leading mobile analytics tools. [link]. Accessed: 2025-08-08.
W3C (1999). Web content accessibility guidelines 2.2. [link]. Accessed: 2025-04-05.
(WAI), W. W. A. I. (2008). Understanding conformance requirements. [link]. Accessed: 2025-04-14.
Xu, Y., Li, Z., Liu, H., e Liu, Y. (2023). Agaa: An android gui accessibility adapter for low vision users. In 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), pages 412–421. IEEE.
Yan, S. e Ramachandran, P. (2019a). The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS), 12(1):1–31.
Yan, S. e Ramachandran, P. (2019b). The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS), 12(1):1–31.
Zhang, M., Liu, H., Chen, C., Gao, G., Li, H., e Zhao, J. (2023a). Accessfxer: Enhancing gui accessibility for low vision users with r-gcn model. IEEE Transactions on Software Engineering.
Zhang, Y., Chen, S., Fan, L., Chen, C., e Li, X. (2023b). Automated and context-aware repair of color-related accessibility issues for android apps. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1255–1267.
Publicado
08/09/2025
Como Citar
RAMOS BARBOZA, Alex Frederico; ELER, Marcelo Medeiros.
VisioAux: a crowdsource approach for accessibility evaluation in mobile devices. In: SIMPÓSIO BRASILEIRO SOBRE FATORES HUMANOS EM SISTEMAS COMPUTACIONAIS (IHC), 24. , 2025, Belo Horizonte/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 446-473.
DOI: https://doi.org/10.5753/ihc.2025.10794.
