How Can Copilot Assist in Creating Accessible Websites? An Empirical Study

Abstract


Introduction: Software developers have used intelligent coding assistants to aid their development efforts. Objective: We investigated GitHub Copilot’s support for creating accessible software applications. We explored its capacity to generate code suggestions that adhere to accessibility best practices during the development of a web application. Methodology: We analyzed the results using an inspection tool and the content of development diaries. Results: Our results indicate that the assistant provides limited accessibility support, exhibiting three distinct behaviors: in some cases, it generates code that adheres to accessibility criteria autonomously, while in others, it requires detailed prompting. Furthermore, in some cases, it does not even provide accessibility support when explicitly requested by the developer.

Keywords: Digital accessibility, Software development, Intelligent assistant

References

Acosta-Vargas, P., Salvador-Acosta, B., Novillo-Villegas, S., Sarantis, D., e SalvadorUllauri, L. (2024). Generative artificial intelligence and web accessibility: Towards an inclusive and sustainable future. Emerging Science Journal, 8(4):1602–1621.

Alshayban, A., Ahmed, I., e Malek, S. (2020). Accessibility issues in android apps: state of affairs, sentiments, and ways forward. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, ICSE ’20, page 1323–1334, New York, NY, USA. Association for Computing Machinery.

Antonelli, H. L., Rodrigues, S. S., Watanabe, W. M., e de Mattos Fortes, R. P. (2018). A survey on accessibility awareness of brazilian web developers. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion, DSAI ’18, page 71–79, New York, NY, USA. Association for Computing Machinery.

Bardzell, S. e Bardzell, J. (2011). Towards a feminist hci methodology: social science, feminism, and hci. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, page 675–684, New York, NY, USA. Association for Computing Machinery.

Barke, S., James, M. B., e Polikarpova, N. (2023). Grounded copilot: How programmers interact with code-generating models. Proc. ACM Program. Lang., 7(OOPSLA1).

Bender, E. M., Gebru, T., McMillan-Major, A., e Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Association for Computing Machinery.

Bird, C., Ford, D., Zimmermann, T., Forsgren, N., Kalliamvakou, E., Lowdermilk, T., e Gazit, I. (2023). Taking fight with copilot: Early insights and opportunities of ai-powered pair-programming tools. Queue, 20(6):35–57.

Borges, J. M. e Araújo, R. D. (2024). Experiences and challenges of a redesign process with the support of an ai assistant on an educational platform. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.

Brazil (1991). Law 8,213, dated july 24, 1991. [link]. Accessed on August 08, 2025.

Brazil (2009). Decree No. 6,949 of August 25, 2009. [link]. Accessed on May 06, 2024.

Cui, Z., Demirer, M., Jaffe, S., Musolff, L., Peng, S., e Salz, T. (2024). The Effects of Generative AI on High Skilled Work: Evidence from Three Field Experiments with Software Developers. SSRN eLibrary.

da Silva, I., Silva, M. P., Teran, L. A., e Mota, M. P. (2024). Desafios da inteligência artificial generativa na construção de sistemas computacionais acessíveis. In Anais do XV Workshop sobre Aspectos da Interação Humano-Computador para a Web Social, pages 19–28, Porto Alegre, RS, Brasil. SBC.

de Souza, C. R. B., Rodríguez-Pérez, G., Basha, M., Yoon, D., e Beschastnikh, I. (2024). The fine balance between helping with your job and taking it: Ai code assistants come to the fore. IEEE Software, 41(6):111–118.

Deque Systems (2024). Glossary | Deque Docs — docs.deque.com. [link]. Accessed on April 23, 2024.

Dohmke, T. (2023). Github copilot x: The ai-powered developer experience. [link]. Accessed on May 29, 2025.

Duarte, E. F., Toledo Palomino, P., Pontual Falcão, T., Porto, G. L. P. M. B., Portela, C. d. S., Ribeiro, D. F., Nascimento, A., Costa Aguiar, Y. P., Souza, M., Moutin Segoria Gasparotto, A., e Maciel Toda, A. (2024). Grandihc-br 2025-2035 - gc6: Implications of artificial intelligence in hci: A discussion on paradigms ethics and diversity equity and inclusion. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems, IHC ’24, New York, NY, USA. Association for Computing Machinery.

Erete, S., Israni, A., e Dillahunt, T. (2018). An intersectional approach to designing in the margins. Interactions, 25(3):66–69.

Ernst, N. A. e Bavota, G. (2022). Ai-driven development is here: Should you worry? IEEE Software, 39(2):106–110.

Freire, A. P., Cardoso, P. C. F., e Salgado, A. d. L. (2024). May we consult chatgpt in our human-computer interaction written exam? an experience report after a professor answered yes. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, IHC ’23, New York, NY, USA. Association for Computing Machinery.

Freire, A. P., Goularte, R., e de Mattos Fortes, R. P. (2007). Techniques for developing more accessible web applications: A survey towards a process classification. In Proceedings of the 25th Annual ACM International Conference on Design of Communication, SIGDOC ’07, page 162–169, New York, NY, USA. ACM.

Freire, A. P., Russo, C. M., e Fortes, R. P. (2008). A survey on the accessibility awareness of people involved in web development projects in brazil. In Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), pages 87–96.

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., e Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Germany (2002). Law on equality for persons with disabilities. [link]. Accessed on February 03, 2025.

GitHub (2025). Getting started with prompts for Copilot Chat - GitHub Docs. [link]. Accessed on Feb 02, 2025.

Guspian, I., Lin, C.-C., Huang, A. Y., e Yang, S. J. (2024). Transforming outreach effects: The impact of generative ai on inclusive education branding and sustainability. Journal of Education For Sustainable Innovation, 2(2):170–179.

Hamza, M., Siemon, D., Akbar, M. A., e Rahman, T. (2024). Human-ai collaboration in software engineering: Lessons learned from a hands-on workshop. In Proceedings of the 7th ACM/IEEE International Workshop on Software-Intensive Business, IWSiB ’24, page 7–14, New York, NY, USA. Association for Computing Machinery.

Henderson-Summet, V., Grinter, R. E., Carroll, J., e Starner, T. (2007). Electronic communication: Themes from a case study of the deaf community. In Human-Computer Interaction–INTERACT 2007: 11th IFIP TC 13 International Conference, Rio de Janeiro, Brazil, September 10-14, 2007, Proceedings, Part I 11, pages 347–360. Springer.

Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., e Wang, H. (2024). Large language models for software engineering: A systematic literature review. ACM Trans. Softw. Eng. Methodol., 33(8).

Kavcic, A. (2005). Software accessibility: Recommendations and guidelines. In EUROCON 2005 - The International Conference on "Computer as a Tool", volume 2, pages 1024–1027.

Keyes, O., Peil, B., Williams, R. M., e Spiel, K. (2020). Reimagining (women’s) health: Hci, gender and essentialised embodiment. ACM Trans. Comput.-Hum. Interact., 27(4).

Kulkarni, M. (2019). Digital accessibility: Challenges and opportunities. IIMB Management Review, 31(1):91–98.

Leite, M. V. R., Scatalon, L. P., Freire, A. P., e Eler, M. M. (2021). Accessibility in the mobile development industry in brazil: Awareness, knowledge, adoption, motivations and barriers. Journal of Systems and Software, 177:110942.

Liang, C. A., Munson, S. A., e Kientz, J. A. (2021). Embracing four tensions in human-computer interaction research with marginalized people. ACM Trans. Comput.-Hum. Interact., 28(2).

Liang, J. T., Yang, C., e Myers, B. A. (2024). A large-scale survey on the usability of ai programming assistants: Successes and challenges. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, New York, NY, USA. Association for Computing Machinery.

Liu, Y., Chen, X., Gao, Y., Su, Z., Zhang, F., Zan, D., Lou, J.-G., Chen, P.-Y., e Ho, T.-Y. (2023). Uncovering and quantifying social biases in code generation. NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.

Mateus, D. A., Silva, C. A., Eler, M. M., e Freire, A. P. (2020). Accessibility of mobile applications: evaluation by users with visual impairment and by automated tools. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, IHC ’20, New York, NY, USA. Association for Computing Machinery.

MoghadasNian, S., MoghadasNian, S., e MoghadasNian, A. (2025). Generative ai in airline tourism: Enhancing personalization with equity and accessibility. In Proceedings of the 4th International Congress on Management, Economy, Humanities and Business Development. Tabriz Islamic Art University, Tabriz, Iran.: English.

Mowar, P., Peng, Y.-H., Steinfeld, A., e Bigham, J. P. (2024). Tab to autocomplete: The effects of ai coding assistants on web accessibility. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility, pages 1–6.

Mowar, P., Peng, Y.-H., Wu, J., Steinfeld, A., e Bigham, J. P. (2025). Codea11y: Making ai coding assistants useful for accessible web development. arXiv preprint arXiv:2502.10884.

Mozilla (2024). Aria - acessibilidade - mdn web docs. [link]. Accessed on April 23, 2024.

Next.js (2024). Next.js by vercel - the react framework. [link]. Accessed on April 23, 2024.

Nguyen, N. e Nadi, S. (2022). An empirical evaluation of github copilot’s code suggestions. In 2022 IEEE/ACM 19th International Conference on Mining Software Repositories (MSR), pages 1–5.

npm, Inc. (2024). pa11y — npmjs.com. [link]. Accessed on May 06, 2024.

Ouyang, S., Zhang, J. M., Harman, M., e Wang, M. (2023). Llm is like a box of chocolates: the non-determinism of chatgpt in code generation. arXiv preprint arXiv:2308.02828.

Ozkaya, I. (2023). The next frontier in software development: Ai-augmented software development processes. IEEE Software, 40(4):4–9.

Pa11y (2023). Pa11y — pa11y.org. [link]. Accessed on May 06, 2024.

Palen, L. e Salzman, M. (2002). Voice-mail diary studies for naturalistic data capture under mobile conditions. In Proceedings of the 2002 ACM conference on Computer supported cooperative work, pages 87–95.

Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., e Karri, R. (2022). Asleep at the keyboard? assessing the security of github copilot’s code contributions. In 2022 IEEE Symposium on Security and Privacy (SP), pages 754–768. IEEE.

Portugal (2004). Law 38/2004, dated august 18, 2004. [link]. Accessed on May 06, 2024.

Rajbhoj, A., Somase, A., Kulkarni, P., e Kulkarni, V. (2024). Accelerating software development using generative ai: Chatgpt case study. In Proceedings of the 17th Innovations in Software Engineering Conference, ISEC ’24, New York, NY, USA. Association for Computing Machinery.

Ross, S. I., Martinez, F., Houde, S., Muller, M., e Weisz, J. D. (2023). The programmer’s assistant: Conversational interaction with a large language model for software development. In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI ’23, page 491–514, New York, NY, USA. Association for Computing Machinery.

Runeson, P. e Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14:131–164.

Schmidt, A. (2023). Speeding up the engineering of interactive systems with generative ai. In Companion Proceedings of the 2023 ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS ’23 Companion, page 7–8, New York, NY, USA. Association for Computing Machinery.

Silva, J. S. e., Gonçalves, R., Branco, F., Pereira, A., Au-Yong-Oliveira, M., e Martins, J. (2019). Accessible software development: a conceptual model proposal. Universal Access in the Information Society, 18:703–716.

Teran, L. A., da Rocha, T. A., da Silva, I. M. M., de Salles, R. B., Melo, G. L. N., Mota, M. P., e de Souza, C. R. B. (2025a). Github copilot diary - researcher 1. [link]. Accessed on August 08, 2025.

Teran, L. A., da Rocha, T. A., da Silva, I. M. M., de Salles, R. B., Melo, G. L. N., Mota, M. P., e de Souza, C. R. B. (2025b). Github copilot diary - researcher 2. [link]. Accessed on August 08, 2025.

Teran, L. A., de Almeida Silva, A. T., Melo, G. L. N., e Mota, M. P. (2021). A study on discovering accessibility issues in the software development process. In CONTECSI International Conference on Information Systems and Technology Management. TECSI.

Terragni, V., Vella, A., Roop, P., e Blincoe, K. (2025). The future of ai-driven software engineering. ACM Trans. Softw. Eng. Methodol. Just Accepted.

USA (2024). Ada.gov: The americans with disabilities act. [link]. Accessed on May 06, 2024.

Whitaker, R. (2016). Mobile a11y. [link]. Accessed on April 21, 2024.

World Wide Web Consortium (2016). User agent accessibility guidelines (uaag) overview. [link]. Accessed on April 21, 2024.

World Wide Web Consortium (2023a). Authoring tool accessibility guidelines (atag) overview. [link]. Accessed on April 21, 2024.

World Wide Web Consortium (2023b). Understanding SC 1.3.1: Info and Relationships. [link]. Accessed on May 05, 2024.

World Wide Web Consortium (2023c). Understanding SC 1.4.3: Contrast (Minimum). [link]. Accessed on May 05, 2024.

World Wide Web Consortium (2023d). Understanding SC 2.4.2: Page Titled. [link]. Accessed on May 05, 2024.

World Wide Web Consortium (2023e). Understanding SC 4.1.2: Name, Role, Value. [link]. Accessed on May 05, 2024.

World Wide Web Consortium (2023f). Understanding understanding sc 2.4.1: Bypass blocks (level a). [link]. Accessed on February 14, 2025.

World Wide Web Consortium (2024a). Introduction to Web Accessibility. [link]. Accessed on May 06, 2024.

World Wide Web Consortium (2024b). Wcag 2 overview. [link]. Accessed on May 06, 2024.

Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., e Aftandilian, E. (2022). Productivity assessment of neural code completion. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, MAPS 2022, page 21–29, New York, NY, USA. Association for Computing Machinery.
Published
2025-09-08
TERAN, Luciano A.; DA ROCHA, Thayssa A.; DA SILVA, Ingrid M. M.; DE SALLES, Rafael B.; MELO, Giselle L. N.; MOTA, Marcelle P.; DE SOUZA, Cleidson R. B.. How Can Copilot Assist in Creating Accessible Websites? An Empirical Study. In: BRAZILIAN SYMPOSIUM ON HUMAN FACTORS IN COMPUTATIONAL SYSTEMS (IHC), 24. , 2025, Belo Horizonte/MG. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2025 . p. 196-221. DOI: https://doi.org/10.5753/ihc.2025.10811.