Pipeline Automatizado para Avaliação de Acessibilidade de Interfaces Android Geradas por LLMs

• Leonardo Quezado UFC
• Ribamar de Souza Martins UFC
• Windson Viana UFC

DOI: https://doi.org/10.5753/webmedia.2026.30777

Resumo

Research evaluating the quality of artifacts generated by Large Language Models (LLMs), including code, user interfaces, and media, has expanded considerably. However, most evaluation remains manual and time-consuming, making it difficult to keep pace with rapidly released model versions. Recent work also suggests that newer models can regress relative to their predecessors, reinforcing the need for continuous and repeatable evaluation. Within this broader challenge, assessing the accessibility of LLM-generated mobile interfaces is particularly relevant. This paper presents an end-to-end automated pipeline focused on structural and semantic properties for Android Jetpack Compose interfaces, integrating code extraction, deterministic repair rules, Android compilation, custom Lint checks, and dynamic Accessibility Test Framework (ATF) execution with minimal human intervention. We evaluated the pipeline with three LLMs (Claude Sonnet 4.6, ChatGPT 5.5, and Gemini 3.1 Pro) across eight representative mobile interface types. Across 48 evaluations, the pipeline achieved 100% compilation success and reported 409 accessibility findings across static and runtime checks (some of which may correspond to the same underlying defect). These preliminary results show that end-to-end automation can support feasible, repeatable, and scalable accessibility assessment of LLM-generated Compose code while substantially reducing the manual effort required to evaluate rapidly evolving models.

Palavras-chave: LLM, Jetpack Compose, Accessibility, Android, Accessibility Test Framework, Pipeline

Referências

Abdulaziz Alshayban, Iftekhar Ahmed, and Sam Malek. 2020. Accessibility issues in Android apps: state of affairs, sentiments, and ways forward. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 1323–1334. DOI: 10.1145/3377811.3380392

Lingjiao Chen, Matei Zaharia, and James Zou. 2023. How Is ChatGPT’s Behavior Changing over Time? arXiv:2307.09009 [cs.AI]. [link].

Sen Chen, Chunyang Chen, Lingling Fan, Mingming Fan, Xian Zhan, and Yang Liu. 2022. Accessible or Not? An Empirical Investigation of Android App Accessibility. arXiv:2203.06422 [cs.SE] [link]

Daniel de Sousa Moraes, Polyana Bezerra da Costa, Antonio JG Busson, José Matheus Carvalho Boaro, Carlos de Salles Soares Neto, and Sergio Colcher. 2023. On the Challenges of Using Large Language Models for NCL Code Generation. In Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia). SBC, 151–156.

Xueying Du, Mingwei Liu, KaixinWang, HanlinWang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2024. Evaluating Large Language Models in Class-Level Code Generation. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Computing Machinery, New York, NY, USA, Article 81, 13 pages. DOI: 10.1145/3597503.3639219

Nayse Fagundes and Leopoldo Teixeira. 2025. Evaluating LLMs for Multimodal GUI Test Generation in Android Applications. In Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST). SBC, 28–36.

Nadeen Fathallah, Daniel Hernández, and Steffen Staab. 2025. AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code. In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility. Association for Computing Machinery, 1–22. DOI: 10.1145/3663547.3746360

Reinaldo Ferraz, Ana Duarte, João Bárbara, Adriano C.M. Pereira, and Wagner Meira. 2023. A platform to check website compliance with web accessibility standards. In Proceedings of the 20th International Web for All Conference (Austin, TX, USA) (W4A ’23). Association for Computing Machinery, New York, NY, USA, 75–78. DOI: 10.1145/3587281.3587289

Google. 2024. Jetpack Compose Accessibility Guide. [link]. Acesso em: ago. 2025.

Google. 2024. Jetpack Compose UI App Development Toolkit. [link]. Acesso em: ago. 2025.

Google. 2026. Accessibility Test Framework for Android. [link]. Acesso em: mai. 2026.

Guilherme Lima, Rodrigo Costa, and Marcio Moreno. 2019. Semantics and multimedia: an introduction to symbolic artificial intelligence applied to multimedia. In Proceedings of the 25th Brazillian Symposium on Multimedia and the Web (Rio de Janeiro, Brazil) (WebMedia ’19). Association for Computing Machinery, New York, NY, USA, 7–9. DOI: 10.1145/3323503.3345031

Matheus Lima, Guido Lemos, and Raoni Kulesza. 2021. A Proposal for an Accessible Videoconferencing Interface for People with Hearing Impairments. In Proceedings of the Brazilian Symposium on Multimedia and the Web. 29–36.

Alberto Dumont Alves Oliveira, Paulo Sérgio Henrique Dos Santos, Wilson Estécio Marcílio Júnior, Wajdi M Aljedaani, Danilo Medeiros Eler, and Marcelo Medeiros Eler. 2023. Analyzing Accessibility Reviews Associated with Visual Disabilities or Eye Conditions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 37, 14 pages. DOI: 10.1145/3544548.3581315

Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv:2302.06590 [cs.SE]. [link].

Diego M. F. Rabelo, Matheus Andrade, Ribamar de Souza Martins, and Windson Viana. 2025. Breaking Barriers in Mobile Accessibility: A Study of LLM-Generated Native Android Interfaces. In Proceedings of the 12th IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft ’25). DOI: 10.1109/MOBILESoft66462.2025.00010

Stack Overflow. 2025. Stack Overflow Developer Survey 2025: AI Agent Out-of-the-Box Tools. [link]. Acesso em: jun. 2026.

Hyunjae Suh, Mahan Tafreshipour, Sam Malek, and Iftekhar Ahmed. 2025. Human or LLM? A Comparative Study on Accessible Code Generation Capability. arXiv:2503.15885 [cs.SE]. [link].

Michael Cheng-Tek Tai. 2020. The impact of artificial intelligence on human society and bioethics. Tzu Chi Medical Journal 32, 4 (Oct–Dec 2020), 339–343. DOI: 10.4103/tcmj.tcmj_71_20

Christopher Vendome, Diana Solano, Santiago Liñán, and Mario Linares-Vásquez. 2019. Can Everyone use my app? An Empirical Study on Accessibility in Android Apps. In 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME). 41–52. DOI: 10.1109/ICSME.2019.00014

Carlos Ventura, Daniel Rabelo, Ticianne Darin, and Windson Viana. 2025. Accessibility Evaluation of LLM-Generated Android Code in React Native. In Anais do XXIV Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (Belo Horizonte/MG). SBC, Porto Alegre, RS, Brasil, 1301–1322. DOI: 10.5753/ihc.2025.10833

Lucas Xavier, Daniel Rabelo, Ribamar Martins, and Windson Viana. 2025. Image-to-UI Accessibility: Assessing ChatGPT’s Effectiveness in Generating Accessible Android Screens from Figma Templates. In Proceedings of the 31st Brazilian Symposium on Multimedia and the Web (Rio de Janeiro/RJ). SBC, Porto Alegre, RS, Brasil, 302–311. DOI: 10.5753/webmedia.2025.15298

Shunguo Yan and P. G. Ramachandran. 2019. The Current Status of Accessibility in Mobile Apps. ACM Trans. Access. Comput. 12, 1, Article 3 (Feb. 2019), 31 pages. DOI: 10.1145/3300176