Uma avaliação de ferramentas de análise de sentimentos aplicadas a comentários da plataforma GitHub
Resumo
O desenvolvimento distribu?do de software tem se tornado frequente e a interação entre os envolvidos, muitas vezes influenciada por aspectos sociais e culturais, reflete no desempenho das equipes. A análise de sentimentos vem sendo empregada para capturar informações subjetivas e obter um maior entendimento das interações dessas equipes. Portanto, é interessante avaliar o desempenho das ferramentas dispon?veis quando aplicadas a esse dom?nio. Neste trabalho nove ferramentas de análise de sentimentos foram avaliadas usando comentários extra?dos da plataforma GitHub e que foram manualmente anotados quanto à polaridade. Os resultados mostraram que a ferramenta SentiStrength se saiu melhor, porém com desempenho médio abaixo de 50%.
Palavras-chave:
GitHub, mineração de repositórios, análise de sentimentos
Referências
Araujo, M., Gonc ̧alves, P., Cha, M., and Benevenuto, F. (2014). ifeel: A system that compares and combines sentiment analysis methods. In Proc. of the 23rd Int. Conf. on World wide web Companion, pages 75–78.
Baccianella, S., Esuli, A., and Sebastiani, F. (2010). Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proc. of the 7th Int. Conf. on Language Resources and Evaluation, pages 2200–2204.
Cambria, E., Speer, R., Havasi, C., and Hussain, A. (2010). Senticnet: A publicly avai- lable semantic resource for opinion mining. In AAAI fall symposium: commonsense knowledge.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psy- chological Measurement, (20):213–220.
Cruz, G., Huzita, E., and Feltrim, V. (2016). Estimating trust in virtual teams - a fra- mework based on sentiment analysis. In Proc. of the 18th Int. Conf. on Enterprise Information Systems (ICEIS 2016), pages 464–471.
Dodds, P. S. and Danforth, C. M. (2010). Measuring the happiness of large-scale written expression: Songs, blogs, and presidents. Journal of Happiness Studies, 11(4):441– 456.
Goncalves, P., Benevenuto, F., and Almeida, V. (2013). O que tweets contendo emoticons podem revelar sobre sentimentos coletivos. In Proc. of the II Brazilian Workshop on Social Network Analysis and Mining (BraSNAM), pages 1–12.
Goncalves, P., Dores, W., and Benevenuto, F. (2012). Panas-t: Uma escala psicometrica para analise de sentimentos no twitter. In Proc. of the I Brazilian Workshop on Social Network Analysis and Mining (BraSNAM).
Guzman, E., Azo ́car, D., and Li, Y. (2014). Sentiment analysis of commit comments in github: An empirical study. In Proc. of the 11th Working Conf. on Mining Software Repositories, pages 352–355.
Herbsleb, J. D. and Moitra, D. (2001). Global software development. IEEE Software, 18(2):16–20.
Jongeling, R., Datta, S., and Serebrenik, A. (2015). Choosing your weapons: On senti- ment analysis tools for software engineering research. In IEEE Int. Conf. on Software Maintenance and Evolution, pages 531–535. IEEE.
Liu, B. (2012). Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers.
Mohammad, S. M., Kiritchenko, S., and Zhu, X. (2013). Nrc-canada: Building the state- of-the-art in sentiment analysis of tweets. Computing Research Repository (CoRR), abs/1308.6242.
Murgia, A., Tourani, P., Adams, B., and Ortu, M. (2014). Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proc. of the 11th Working Conf. on Mining Software Repositories, pages 262–271.
Nielsen,F.A ̊.(2011).A new anew:Evaluation of a word list for sentiment analysis in microblogs. Computing Research Repository (CoRR), abs/1103.2903.
O’Conchuir, E., Holmstrom, H., Agerfalk, P., and Fitzgerald, B. (2006). Exploring the assumed benefits of global software development. In Int. Conf. on Global Software Engineering, pages 159–168.
Park, J., Barash, V., Fink, C., and Cha, M. (2013). Emoticon style: Interpreting differences in emoticons across cultures. In Proc. of the 7th Int. AAAI Conf. on Weblogs and Social Media.
Sengupta, B., Chandra, S., and Sinha, V. (2006). A research agenda for distributed soft- ware development. In Int. Conf. on Software Engineering. ACM.
Sinha, V., Lazar, A., and Sharif, B. (2016). Analyzing developer sentiment in commit logs. In Proc. of the 13th Int. Conf. on Mining Software Repositories, pages 520–523.
Thelwall, M. (2013). Heart and soul: Sentiment strength detection in the social web with sentistrength. Proceedings of the CyberEmotions, pages 1–14.
Tourani, P., Jiang, Y., and Adams, B. (2014). Monitoring sentiment in open source mailing lists-exploratory study on the apache ecosystem. In Proc. of the 2014 Conf. of the Center for Advanced Studies on Collaborative Research, pages 74–95.
Watson, D., Clark, L. A., and Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the panas scales. Journal of personality and social psychology, 54(6):1063.
Baccianella, S., Esuli, A., and Sebastiani, F. (2010). Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proc. of the 7th Int. Conf. on Language Resources and Evaluation, pages 2200–2204.
Cambria, E., Speer, R., Havasi, C., and Hussain, A. (2010). Senticnet: A publicly avai- lable semantic resource for opinion mining. In AAAI fall symposium: commonsense knowledge.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psy- chological Measurement, (20):213–220.
Cruz, G., Huzita, E., and Feltrim, V. (2016). Estimating trust in virtual teams - a fra- mework based on sentiment analysis. In Proc. of the 18th Int. Conf. on Enterprise Information Systems (ICEIS 2016), pages 464–471.
Dodds, P. S. and Danforth, C. M. (2010). Measuring the happiness of large-scale written expression: Songs, blogs, and presidents. Journal of Happiness Studies, 11(4):441– 456.
Goncalves, P., Benevenuto, F., and Almeida, V. (2013). O que tweets contendo emoticons podem revelar sobre sentimentos coletivos. In Proc. of the II Brazilian Workshop on Social Network Analysis and Mining (BraSNAM), pages 1–12.
Goncalves, P., Dores, W., and Benevenuto, F. (2012). Panas-t: Uma escala psicometrica para analise de sentimentos no twitter. In Proc. of the I Brazilian Workshop on Social Network Analysis and Mining (BraSNAM).
Guzman, E., Azo ́car, D., and Li, Y. (2014). Sentiment analysis of commit comments in github: An empirical study. In Proc. of the 11th Working Conf. on Mining Software Repositories, pages 352–355.
Herbsleb, J. D. and Moitra, D. (2001). Global software development. IEEE Software, 18(2):16–20.
Jongeling, R., Datta, S., and Serebrenik, A. (2015). Choosing your weapons: On senti- ment analysis tools for software engineering research. In IEEE Int. Conf. on Software Maintenance and Evolution, pages 531–535. IEEE.
Liu, B. (2012). Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers.
Mohammad, S. M., Kiritchenko, S., and Zhu, X. (2013). Nrc-canada: Building the state- of-the-art in sentiment analysis of tweets. Computing Research Repository (CoRR), abs/1308.6242.
Murgia, A., Tourani, P., Adams, B., and Ortu, M. (2014). Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proc. of the 11th Working Conf. on Mining Software Repositories, pages 262–271.
Nielsen,F.A ̊.(2011).A new anew:Evaluation of a word list for sentiment analysis in microblogs. Computing Research Repository (CoRR), abs/1103.2903.
O’Conchuir, E., Holmstrom, H., Agerfalk, P., and Fitzgerald, B. (2006). Exploring the assumed benefits of global software development. In Int. Conf. on Global Software Engineering, pages 159–168.
Park, J., Barash, V., Fink, C., and Cha, M. (2013). Emoticon style: Interpreting differences in emoticons across cultures. In Proc. of the 7th Int. AAAI Conf. on Weblogs and Social Media.
Sengupta, B., Chandra, S., and Sinha, V. (2006). A research agenda for distributed soft- ware development. In Int. Conf. on Software Engineering. ACM.
Sinha, V., Lazar, A., and Sharif, B. (2016). Analyzing developer sentiment in commit logs. In Proc. of the 13th Int. Conf. on Mining Software Repositories, pages 520–523.
Thelwall, M. (2013). Heart and soul: Sentiment strength detection in the social web with sentistrength. Proceedings of the CyberEmotions, pages 1–14.
Tourani, P., Jiang, Y., and Adams, B. (2014). Monitoring sentiment in open source mailing lists-exploratory study on the apache ecosystem. In Proc. of the 2014 Conf. of the Center for Advanced Studies on Collaborative Research, pages 74–95.
Watson, D., Clark, L. A., and Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the panas scales. Journal of personality and social psychology, 54(6):1063.
Publicado
25/10/2016
Como Citar
PORTOLESE, Giuseppe; DA CRUZ, Guilherme; HUZITA, Elisa; FELTRIM, Valéria.
Uma avaliação de ferramentas de análise de sentimentos aplicadas a comentários da plataforma GitHub. In: WORKSHOP SOBRE ASPECTOS SOCIAIS, HUMANOS E ECONÔMICOS DE SOFTWARE (WASHES), 1. , 2016, Maceió.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2016
.
p. 21-30.
ISSN 2763-874X.
