What Public Repositories Reveal: Inferring Developer Skills from GitHub
Resumo
Public software repositories record evidence about development activities, but they represent only part of a person’s trajectory. This article presents SSM (Source Signal Miner), a method that organizes public GitHub data into five observable indicators: public participation, textual polarity, technological diversity, contribution continuity, and public activity intensity. The method also assigns a predominant technical area and generates a report associating the results with the amount and coverage of the evidence used. These indicators describe patterns of public activity and do not constitute direct measures of competence, productivity, or professional performance. The evaluation was conducted through two complementary studies. A perceptual study with nine Information Systems students analyzed reports produced from their own profiles. Separately, the technical classification component was evaluated using a labeled dataset of 1,240 real profiles, from which 372 were reserved for the final test. The mean perceived adherence of report components, on a normalized scale from 0 to 1, ranged from 0.63 to 0.73. The ability of the report to represent content available in the public profile received a mean score of 0.76, while the ability of the public profile itself to represent complete experience received a mean score of 0.42. In the independent test, the classifier achieved an accuracy of 88.17% and a macro F1-score of 0.88. Results indicate that public signals can support an initial characterization, provided that limitations in data coverage are made explicit and results are combined with other sources of information.
Referências
Lingfeng Bao, Xin Xia, David Lo, and Gail C. Murphy. 2019. A Large Scale Study of Long-Time Contributor Prediction for GitHub Projects. IEEE Trans. Software Eng. 47 (2019), 1277–1298. DOI: 10.1109/tse.2019.2918536
Fabio Calefato, Filippo Lanubile, Federico Maiorano, and Nicole Novielli. 2018. Sentiment Polarity Detection for Software Development. Empirical Software Engineering 23, 3 (2018), 1352–1382. DOI: 10.1007/s10664-017-9542-0
Chiara Del Vescovo, Davide Taibi, and Andrea Janes. 2021. Profiling Developers Through the Analysis of GitHub Public Repositories. In Proc. IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). DOI: 10.1109/msr52588.2021.00059
Thomas Fritz, Jingwen Ou, Gail C. Murphy, and Thomas Zimmermann. 2010. A Degree-of-Knowledge Model to Capture Source Code Familiarity. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering. ACM, 385–394.
Emitza Guzmán, David Azócar, and Yang Li. 2014. Sentiment Analysis of Commit Comments in GitHub: An Empirical Study. In Proc. 11th Working Conf. on Mining Software Repositories. 352–355. DOI: 10.1145/2597073.2597118
Clayton J. Hutto and Eric Gilbert. 2014. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. In Proc. 8th International AAAI Conf. on Weblogs and Social Media.
Robbert Jongeling, Subhajit Datta, and Alexander Serebrenik. 2015. Choosing Your Weapons: On Sentiment Analysis Tools for Software Engineering Research. In 2015 IEEE International Conference on Software Maintenance and Evolution. IEEE, 531–535. DOI: 10.1109/ICSM.2015.7332510
Huzefa Kagdi, Maen Hammad, and Jonathan I. Maletic. 2008. Who Can Help Me with This Source Code Change?. In 2008 IEEE International Conference on Software Maintenance. IEEE, 157–166. DOI: 10.1109/ICSM.2008.4658064
Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M. German, and Daniela Damian. 2014. The Promises and Perils of Mining GitHub. In Proc. MSR. 92–101. DOI: 10.1145/2597073.2597074
Jenny T. Liang, Thomas Zimmermann, and Denae Ford. 2022. Towards Mining OSS Skills fromGitHubActivity. In Proceedings of the 44th International Conference on Software Engineering: New Ideas and Emerging Results. 106–110. DOI: 10.1145/3510455.3512772
Jenny T. Liang, Thomas Zimmermann, and Denae Ford. 2022. Understanding Skills for OSS Communities on GitHub. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. DOI: 10.1145/3540250.3549082
Audris Mockus and James D. Herbsleb. 2002. Expertise Browser: A Quantitative Approach to Identifying Expertise. In Proceedings of the 24th International Conference on Software Engineering. ACM, 503–512. DOI: 10.1145/581339.581401
João Eduardo Montandon, Luciana Lourdes Silva, and Marco Tulio Valente. 2019. Identifying Experts in Software Libraries and Frameworks among GitHub Users. In Proceedings of the 16th International Conference on Mining Software Repositories. 276–287. DOI: 10.1109/MSR.2019.00054
João Eduardo Montandon, Luciana Lourdes Silva, and Marco Tulio Valente. 2021. Mining the Technical Roles of GitHub Users. Information and Software Technology 131 (2021), 106485.
Nicole Novielli, Daniela Girardi, and Filippo Lanubile. 2018. A Benchmark Study on Sentiment Analysis for Software Engineering Research. In Proceedings of the 15th International Conference on Mining Software Repositories. ACM, 364–375. DOI: 10.1145/3196398.3196403
Edson Oliveira, Eduardo Fernandes, Igor Steinmacher, Marco Cristo, Tayana Conte, and Alessandro Garcia. 2020. Code and Commit Metrics of Developer Productivity: A Study on Team Leaders Perceptions. Empirical Software Engineering 25 (2020), 2519–2549. DOI: 10.1007/s10664-020-09820-z
Naveen Raman, Minxuan Cao, Yulia Tsvetkov, Christian Kästner, and Bogdan Vasilescu. 2020. Stress and Burnout in Open Source: Toward Finding, Understanding, and Mitigating Unhealthy Interactions. In 2020 IEEE/ACM 42nd ICSE-NIER. 57–60. DOI: 10.1145/3377816.3381732
Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Raula Gaikovina Kula, Norihiro Yoshida, Hajimu Iida, and Ken ichi Matsumoto. 2015. Who Should Review My Code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review. In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering. IEEE, 141–150. DOI: 10.1109/SANER.2015.7081824
Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in Software Engineering. Springer. DOI: 10.1007/978-3-642-29044-2
Wenxin Xiao, Hao He, Wenxin Xu, Yuxia Zhang, and Minghui Zhou. 2023. How Early Participation Determines Long-Term Sustained Activity in GitHub Projects?. In Proc. 31st ACM ESEC/FSE. DOI: 10.1145/3611643.3616349
Yishuo Zhang, Nayyar Zaidi, Jiangning Zhou, and Gang Li. 2021. GANBLR: A Tabular Data Generation Model. In Proc. IEEE International Conference on Data Mining (ICDM). 181–190. DOI: 10.1109/icdm51629.2021.00103
