Hate Speech Detection in Portuguese with Naïve Bayes, SVM, MLP and Logistic Regression
Resumo
Even though social networks can provide free space for discussing ideas, people can also use them to propagate hate speech and, given the amount of written material in such networks, it becomes necessary to rely on automatic methods for identifying this problem. In this work, we set out to verify the use of some classic Machine Learning algorithms for the task of hate speech detection in tweets written in Portuguese, by testing four different models (SVM, MLP, Logistic Regression and Naïve Bayes) with different configurations. Results show that these algorithms produce better results (in terms of micro-averaged F1 score) than the LSTM used for benchmark, being also competitive to other results by the related literature
Palavras-chave:
Hate Speech, Twitter, Neural Network, SVM, MLP, Naïve Bayes, NLP
Referências
Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F. M., Rosso, P., and Sanguinetti, M. (2019). SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, USA.
Bergstra, J. and Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research, 13:281–305.
Bosco, C., Felice, D., Poletto, F., Sanguinetti, M., and Maurizio, T. (2018). Overview of the evalita 2018 hate speech detection task. In EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, volume 2263. 7 A choice made by [Fortuna et al. 2019] which we followed to allow for a comparison to be made.
Byrd, R. H., Lu, P., Nocedal, J., and Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 16(5):1190– 1208.
Collobert, R. and Weston, J. (2009). Deep learning in natural language processing. Tutorial at NIPS.
Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., and Lin, C.-J. (2008). Liblinear: A library for large linear classification. Journal of machine learning research, 9:1871– 1874.
Fersini, E., Nozza, D., and Rosso, P. (2018a). Overview of the evalita 2018 task on automatic misogyny identification (ami). EVALITA Evaluation of NLP and Speech Tools for Italian, 12:59.
Fersini, E., Rosso, P., and Anzovino, M. (2018b). Overview of the task on automatic misogyny identification at ibereval 2018. In IberEval@ SEPLN.
Fortuna, P., da Silva, J. R., Wanner, L., Nunes, S., et al. (2019). A hierarchically-labeled portuguese hate speech dataset. In Proceedings of the Third Workshop on Abusive Language Online.
Fortuna, P. and Nunes, S. (2018). A survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 51(4):1–30.
Garmer, M., Lemon, J., Fellows, I., and Singh, S. (2014). Various coefficients of interrater reliability and agreement.
Han, J., Pei, J., and Kamber, M. (2011). Data mining: concepts and techniques. Elsevier.
Hasanuzzaman, M., Dias, G., and Way, A. (2017). Demographic word embeddings for racism detection on twitter.
Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Mulki, H., Haddad, H., Ali, C. B., and Alshabani, H. (2019). L-hsab: A levantine twitter dataset for hate speech and abusive language. In Proceedings of the Third Workshop on Abusive Language Online.
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016). Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web.
Ptaszynski, M., Pieciukiewicz, A., and Dybała, P. (2019). Results of the poleval 2019 shared task 6: First dataset and open shared task for automatic cyberbullying detection in polish twitter. Proceedings ofthePolEval2019Workshop.
Rajaraman, A. and Ullman, J. D. (2011). Mining of massive datasets. Cambridge.
Saha, P., Mathew, B., Goyal, P., and Mukherjee, A. (2018). Hateminers : Detecting hate speech against women. CoRR, abs/1812.06700.
Wiegand, M., Siegel, M., and Ruppenhofer, J. (2018). Overview of the germeval 2018 shared task on the identification of offensive language.
Bergstra, J. and Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research, 13:281–305.
Bosco, C., Felice, D., Poletto, F., Sanguinetti, M., and Maurizio, T. (2018). Overview of the evalita 2018 hate speech detection task. In EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, volume 2263. 7 A choice made by [Fortuna et al. 2019] which we followed to allow for a comparison to be made.
Byrd, R. H., Lu, P., Nocedal, J., and Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 16(5):1190– 1208.
Collobert, R. and Weston, J. (2009). Deep learning in natural language processing. Tutorial at NIPS.
Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., and Lin, C.-J. (2008). Liblinear: A library for large linear classification. Journal of machine learning research, 9:1871– 1874.
Fersini, E., Nozza, D., and Rosso, P. (2018a). Overview of the evalita 2018 task on automatic misogyny identification (ami). EVALITA Evaluation of NLP and Speech Tools for Italian, 12:59.
Fersini, E., Rosso, P., and Anzovino, M. (2018b). Overview of the task on automatic misogyny identification at ibereval 2018. In IberEval@ SEPLN.
Fortuna, P., da Silva, J. R., Wanner, L., Nunes, S., et al. (2019). A hierarchically-labeled portuguese hate speech dataset. In Proceedings of the Third Workshop on Abusive Language Online.
Fortuna, P. and Nunes, S. (2018). A survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 51(4):1–30.
Garmer, M., Lemon, J., Fellows, I., and Singh, S. (2014). Various coefficients of interrater reliability and agreement.
Han, J., Pei, J., and Kamber, M. (2011). Data mining: concepts and techniques. Elsevier.
Hasanuzzaman, M., Dias, G., and Way, A. (2017). Demographic word embeddings for racism detection on twitter.
Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Mulki, H., Haddad, H., Ali, C. B., and Alshabani, H. (2019). L-hsab: A levantine twitter dataset for hate speech and abusive language. In Proceedings of the Third Workshop on Abusive Language Online.
Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016). Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web.
Ptaszynski, M., Pieciukiewicz, A., and Dybała, P. (2019). Results of the poleval 2019 shared task 6: First dataset and open shared task for automatic cyberbullying detection in polish twitter. Proceedings ofthePolEval2019Workshop.
Rajaraman, A. and Ullman, J. D. (2011). Mining of massive datasets. Cambridge.
Saha, P., Mathew, B., Goyal, P., and Mukherjee, A. (2018). Hateminers : Detecting hate speech against women. CoRR, abs/1812.06700.
Wiegand, M., Siegel, M., and Ruppenhofer, J. (2018). Overview of the germeval 2018 shared task on the identification of offensive language.
Publicado
20/10/2020
Como Citar
SILVA, Adriano; ROMAN, Norton.
Hate Speech Detection in Portuguese with Naïve Bayes, SVM, MLP and Logistic Regression. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 17. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 1-12.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2020.12112.
