Offensive Comments in the Brazilian Web: a dataset and baseline results

• Rogers Prates de Pelle UFRGS
• Viviane P. Moreira UFRGS

DOI: https://doi.org/10.5753/brasnam.2017.3260

Resumo

Brazilian Web users are among the most active in social networks and very keen on interacting with others. Offensive comments, known as hate speech, have been plaguing online media and originating a number of lawsuits against companies which publish Web content. Given the massive number of user generated text published on a daily basis, manually filtering offensive comments becomes infeasible. The identification of offensive comments can be treated as a supervised classification task. In order to obtain a model to classify comments, an annotated dataset containing positive and negative examples is necessary. The lack of such a dataset in Portuguese, limits the development of detection approaches for this language. In this paper, we describe how we created annotated datasets of offensive comments for Portuguese by collecting news comments on the Brazilian Web. In addition, we provide classification results achieved by standard classification algorithms on these datasets which can serve as baseline for future work on this topic.

Referências

Michael Chau and Jennifer Xu. Mining communities and their relationships in blogs: A study of online hate groups. International Journal of Human-Computer Studies, 65(1): 57–70, 2007.

Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. Detecting offensive language in social media to protect adolescent online safety. In Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confernece on Social Computing (SocialCom), pages 71–80, 2012.

Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. Hate speech detection with comment embeddings. In Proceedings of the 24th International Conference on World Wide Web Companion, pages 29–30, 2015.

J.L. Fleiss. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378–382, 1971.

Irene Kwok and Yuzhou Wang. Locate the hate: Detecting tweets against blacks. In Twenty-Seventh AAAI Conference on Artificial Intelligence, 2013.

Shuhua Liu and Thomas Forss. Text classification models for web content filtering and online safety. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pages 961–968, 2015.

NoavaS/B. The Webcertain Global Search & Social Report, 2016. URL [link].

Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web, pages 145–153, 2016.

John T. Nockleby. Hate speech. In Encyclopedia of the American Constitution (2nd ed.,edited by Leonard W. Levy, Kenneth L. Karst et al., New York: Macmillan, 2000), pages 1277–1279, 2000.

Amir H Razavi, Diana Inkpen, Sasha Uritsky, and Stan Matwin. Offensive language detection using multi-level classification. In Advances in Artificial Intelligence, pages 16–27. 2010.

Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis. In Proceedings of NLP4CMC III: 3rd Workshop on Natural Language Processing for Computer-Mediated Communication, volume 17 of Bochumer Linguistische Arbeitsberichte, pages 6–9, 2016.

Leandro Araújo Silva, Mainack Mondal, Denzil Correa, Fabrício Benevenuto, and Ingmar Weber. Analyzing the targets of hate in online social media. In Proceedings of the Tenth International Conference on Web and Social Media, pages 687–690, 2016.

Sara Owsley Sood, Judd Antin, and Elizabeth F Churchill. Using crowdsourcing to improve profanity detection. In AAAI Spring Symposium: Wisdom of the Crowds, volume 12, pages 69–74, 2012.

I Ting, Shyue-Liang Wang, Hsing-Miao Chi, Jyun-Sing Wu, et al. Content matters: A study of hate groups detection based on social networks analysis and web mining. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pages 1196–1201, 2013.

William Warner and Julia Hirschberg. Detecting hate speech on the world wide web. In Proceedings of the Second Workshop on Language in Social Media, pages 19–26, 2012.

Zeerak Waseem. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 138–142, November 2016.

Webcertain. The Webcertain Global Search & Social Report, 2015. URL [link].

Ellery Wulczyn, Nithum Thain, and Lucas Dixon. Ex machina: Personal attacks seen at scale. CoRR, abs/1610.08914, 2016.

Guang Xiang, Bin Fan, Ling Wang, Jason Hong, and Carolyn Rose. Detecting offensive tweets via topical feature discovery over a large scale twitter corpus. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, pages 1980–1984, 2012.