LLMs in Practice: Results from a Survey with Software Development Professionals in Brazil
Resumo
Research Context: The emergence of Large Language Models (LLMs) has been transforming the development of information systems, directly impacting activities across the system life cycle. Understanding how these technologies affect work practices is essential for advancing the Information Systems (IS) field and for guiding their adoption by organizations. Scientific and/or Practical Problem: Despite the growing use of LLMs in information systems development, there is still limited knowledge on how professionals across different roles, levels of experience, and work modalities perceive the impact of generative AI on daily activities, decision-making, and team performance. Proposed Solution and/or Analysis: This study investigates the perceptions and experiences of Brazilian software developers regarding the integration of Generative AI (LLMs) into their professional practice. The analysis aims to identify which activities are most impacted by AI, the most frequently adopted tools, and the level of trust professionals place in AI-generated outputs. Related IS Theory: The study aligns with the Systemic and Socially Aware Perspective for Information Systems challenge of GranDSI-BR, by examining how emerging technologies reshape sociotechnical systems and the dynamics of professional work. Research Method: An online survey was conducted with 94 professionals in the field of information systems development, covering diverse profiles. Data were collected through customized Likert scales, multiple-choice checkboxes, and closed-ended questions. Summary of Results: Findings highlight that, although the adoption of LLMs significantly increases developers’ productivity, there is still a moderate level of trust in the outputs and the absence of consolidated governance policies. Contributions and Impact to IS area: The results provide both theoretical and practical insights into the adoption of generative AI in information systems development, offering valuable guidance for researchers, managers, and educators seeking to integrate these technologies into IS safely and ethically.
Referências
Boscarioli, C., de Araujo, R. M., and Maciel, R. S., editors (2017). I GranDSI-BR: Grand Research Challenges in Information Systems in Brazil 2016-2026. Sociedade Brasileira de Computação (SBC), Brazil. Edited by the Special Commission of Information Systems (CE-SI).
Brynjolfsson, E. and McAfee, A. (2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. W. W. Norton & Company.
Brynjolfsson, E. and McAfee, A. (2017). Machine, Platform, Crowd: Harnessing Our Digital Future. W. W. Norton & Company.
Casas, R., Llovet, M., Vilas, D., and Sánchez, A. (2024). The rise of the ai engineer: A new role in the software development landscape. In Proceedings of the 2024 International Conference on Software Engineering (ICSE), pages 1–12. ACM.
Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4):1165–1188.
Chen, X., Gao, C., Chen, C., Zhang, G., and Liu, Y. (2025). An empirical study on challenges for llm application developers. ACM Trans. Softw. Eng. Methodol., 34(7).
Creswell, J. W. and Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE Publications, 5 edition.
Davenport, T. H. (2014). What is an analytic leader? Harvard Business Review, 92(12):57–62.
Fonseca, J. S. d. and Martins, G. d. A. (1996). Curso de Estatística. Atlas.
Gao, C., Hu, X., Gao, S., Xia, X., and Jin, Z. (2025). The current challenges of software engineering in the era of large language models. ACM Trans. Softw. Eng. Methodol., 34(5).
Gil, A. C. (2002). Como Elaborar Projetos de Pesquisa. Atlas, 4 edition.
Goyal, A. et al. (2024). Are large language models the ultimate code generation tool? a comprehensive study on efficacy and challenges. IEEE Software, 41(1):1–10.
Gupta, A., Chen, Y., Raghunathan, A., Mirhoseini, A., and Chen, J. (2022). An empirical assessment of the capabilities of large language models for automated code generation. arXiv preprint arXiv:2203.11183.
Hoffman, R. R., Muge, J. C., and Klein, G. (2018). A matter of trust: Why we need explainable ai. IEEE Intelligent Systems, 33(4):39–55.
Laudon, K. C. and Laudon, J. P. ([Ano da sua edição utilizada]). Sistemas de Informação Gerenciais. Pearson, [Cidade/País], [número da sua edição utilizada] edition.
Li, Y., Zhang, S., Ma, X., Wang, B., Liu, J., Li, X., Xiao, L., He, H., and Xu, Y. (2023). An empirical study on the security of code generated by github copilot. IEEE Transactions on Software Engineering, 49(10):3863–3882.
Minsky, M. L. (1968). Semantic Information Processing. The MIT Press, Cambridge, MA.
Newell, A. and Simon, H. A. (1976). Computer science as empirical inquiry: Symbols and search. Communications of the ACM, 19(3):113–126.
Pechuma, S., Lertsriwatana, K., Laohaprapanon, T., Jaruwat, P., Inpa, W., and Treeprapincha, T. (2023). Exploring the usage of large language models in software development. In Proceedings of the 2023 20th International Conference on Technology and Education (ICTE), pages 1–6. IEEE.
Perry, B. and Uuk, R. (2019). Ai governance and the policymaking process: key considerations for reducing ai risk. Big Data and Cognitive Computing, 3(2):3–12.
Power, D. J. (2002). Decision Support Systems: Concepts and Resources for Managers. Quorum Books, Westport, CT.
Russell, S. J. and Norvig, P. (2020). Artificial Intelligence: A Modern Approach. Pearson Education, Hoboken, NJ, 4 edition.
Vaithilingam, P., Zhang, T., Gauthier, P., Sridhara, G., Bunescu, R., and Liu, D. (2022). More than code: How developers use and understand ai code generation tools. In Proceedings of the ACM/IEEE International Conference on Software Engineering (ICSE), pages 555–567.
Vieira, M. (2025). Leveraging llms for trustworthy software engineering: Insights and challenges. Computer, 58:79–90.
Yadav, S., Kaushik, A., and Qureshi, A. M. (2025). Thematic analysis of expert opinions on the use of large language models in software development. Applied AI Letters, 6(3):e127. e127 AAIL-2025-014.R2.
