Peer assessment of an individual's skills in a software project team: An extension of a systematic literature mapping from the perspective of tools

• Matheus Z. Souza State University of Santa Catarina http://orcid.org/0000-0001-5435-3352
• Nicolle B. Asquino State University of Santa Catarina https://orcid.org/0009-0002-6499-8132
• Avanilde Kemczinski State University of Santa Catarina https://orcid.org/0000-0001-7671-5457

DOI: https://doi.org/10.5753/sbie.2024.242308

Abstract

This study builds upon the Systematic Literature Mapping (SLM) conducted by [Santos et al. 2020], aiming to identify new tools to enhance the features of the “Peer Skills Assessment System” (PSAS) framework, thereby contributing to the evolution of competency assessment practices in software teams. Using the methodology proposed by [Kitchenham et al. 2004], the study was conducted in four stages, from planning to data analysis. The results reveal that formative and summative assessments are the most commonly used types of peer evaluation, with peer ranking and co-evaluation predominating. The questionnaire was identified as the most widely employed tool, leading to its implementation in the PSAS framework.

Keywords: Competency Assessment, Assessment Tools, Peer Evaluation, Questionnaire

References

Anson, R. e Goodman, J. (2013). A peer assessment system to improve student team experiences. Journal of Education for Business, 89(1):27–34.

Bernardo, J. H., Jr, J. J. D., e Cunha, J. A. (2016). Atribuições e competências individuais do scrum master: Um estudo exploratório. In Anais do XII Simpósio Brasileiro de Sistemas de Informação, pages 470–477, Porto Alegre, RS, Brasil. SBC.

Biernacki, P. e Waldorf, D. (1981). Snowball sampling: Problems and techniques of chain referral sampling. Sociological Methods and Research, 10(2):141–163.

Brodie, L. e Gibbings, P. (2009). Comparison of pbl assessment rubrics. Proceedings of the Research in Engineering Education Symposium, page 9.

Correa, A. C. C. (2022). Um [re]design centrado no usuário da visualização de informação no contexto da avaliação de competências. Master’s thesis, Universidade do Estado de Santa Catarina.

Friess, W. A. e Goupee, A. J. (2020). Using continuous peer evaluation in team-based engineering capstone projects: A case study. IEEE Transactions on Education, 63(2):82–87.

Garcia-Souto, M., Azma, Y., Grammenos, R., Kador, T., Striolo, C., Whyndham, M., Vogel, M., Richardson, M., Gibson, A., Britton, J., e Robinson, T. Hughes, G. (2020). Individual peer assessment of contribution to group work (ipac): Key points and recommendations. pages 1553–1565.

Garcia-Souto, M., Hughes, G., Gibson, A., e Correnden, A. (2017). 360 degree peer assessment to train engineering students in giving good quality feedback. pages 1276–1283.

Garcia-Souto, M. d. P. (2019). Making assessment of group work fairer and more insightful for students and time-efficient for staff with the new ipac software. pages 8636–8641.

Grammenos, R., Garcia-Souto, M., e Chester, I. G. L. A. (2019). Peer assessment of individual contribution in group work: a student perspective. pages 1594–1606.

Kemppainen, A., Hamlin, A., Barron, M., e Raber, M. (2022). Comparing team evaluation software (team+ and catme). pages 1–6.

Kitchenham, B., Dyba, T., e Jorgensen, M. (2004). Evidence-based software engineering. pages 273–281.

Mathias, G. e Dorneles, C. (2021). qfex - um crawler para busca e extração de questionários de pesquisa em documentos html. In Anais do III Dataset Showcase Workshop, pages 1–10, Porto Alegre, RS, Brasil. SBC.

Morales-Trujillo, M. E., Galster, M., Gilson, F., e Matthews, M. (2021). A three-year study on peer evaluation in a software engineering project course. IEEE Transactions on Education, 65(3):409–418.

Ohland, M. W., L., L. M., J., W. D., G., B. L., Felder, R. M., J., F. C., Layton, R. A., R., P. H., e G., S. D. (2012). The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self-and peer evaluation. Academy of Management Learning Education, 11(4):609–630.

Poon, J. K. L. (2011). Students’ perceptions of peer evaluation in project work. pages 87–94.

Santos, T. d. R. e Kemczinski, A. (2020). A framework for peer assessment of an individuals skills in a software projects team.

Santos, T. d. R., Pazini, E. d. P., e Kemczinski, A. (2020). Um mapeamento sistemático da literatura sobre a avaliação em pares das competências de um indivíduo em uma equipe de projetos de software. 13:98–131.

Souza, R. e Dorneles, C. (2017). Analisando a eficácia do modelo vetorial de busca na ordenação de questionários. In Anais do XIII Simpósio Brasileiro de Sistemas de Informação, pages 563–570, Porto Alegre, RS, Brasil. SBC.

Strode, D., Dingsoyr, T., e Lindsjorn, Y. (2022). A teamwork effectiveness model for agile software development. Empirical Software Engineering, 27(2):56.

Team+ (2022). Instructor’s guide for the team+ system.

Wang, J., Imbrie, P. K., e Lin, J. J. (2011). A feedback system for peer evaluation of engineering student teams to enhance team effectiveness. pages S4C–1–S4C–5.