Preliminary Evidence on the Use of Chatbots to Support Junior Software Professionals in Process Guidance
Resumo
This study explores whether AI-powered chatbots can enhance task performance and information access for junior professionals. A controlled experiment compared two groups: one used a chatbot, the other a wiki with expert support. Participants completed three tasks of varying difficulty and answered a feedback questionnaire. Preliminary results show that the chatbot group performed better, with statistically significant differences. They also perceived the chatbot as more accessible and efficient, despite noting issues with answer contextualization. These findings suggest chatbots are promising tools for onboarding and learning, though improvements are needed to handle more complex scenarios effectively.
Palavras-chave:
Artificial Intelligence, Chatbots, Software Professionals
Referências
Basili, V. R., Caldiera, G. and Rombach, H. D. (1994). Goal/question/metric approach. In Encyclopedia of Software Engineering, volume 1, pages 528–532. John Wiley & Sons.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences.. Lawrence Erlbaum Associates.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. In Psychometrika, volume 16, pages 297–334. Springer.
Santos, Í., Soares, J. R. and Silva, P. N. (2024). Software solutions for newcomers’ onboarding in software projects: A systematic literature review. In Information & Software Technology, volume 177, page 107568. Elsevier.
Shapiro, S. S. and Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). In Biometrika, volume 52, pages 591–609. Oxford University Press.
Soares, J. R. and Silva, P. N. (2024). Panorama da pesquisa sobre chatbots no Brasil. In Biblos: Revista do Instituto de Ciências Humanas e da Informação, volume 38, number 1, pages 199–218. Universidade Federal do Rio Grande.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences.. Lawrence Erlbaum Associates.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. In Psychometrika, volume 16, pages 297–334. Springer.
Santos, Í., Soares, J. R. and Silva, P. N. (2024). Software solutions for newcomers’ onboarding in software projects: A systematic literature review. In Information & Software Technology, volume 177, page 107568. Elsevier.
Shapiro, S. S. and Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). In Biometrika, volume 52, pages 591–609. Oxford University Press.
Soares, J. R. and Silva, P. N. (2024). Panorama da pesquisa sobre chatbots no Brasil. In Biblos: Revista do Instituto de Ciências Humanas e da Informação, volume 38, number 1, pages 199–218. Universidade Federal do Rio Grande.
Publicado
13/08/2025
Como Citar
NEVES, Fernando Rafael Serra; ANDRADE, João Victor Machado de; FURTADO, Julio Cezar Costa.
Preliminary Evidence on the Use of Chatbots to Support Junior Software Professionals in Process Guidance. In: ESCOLA REGIONAL DE INFORMÁTICA NORTE 2 (ERIN 2), 18. , 2025, Macapá/AP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 7-12.
DOI: https://doi.org/10.5753/erin.2025.15891.
