Beyond Systematic Bias: Investigating Gender Differences in Portuguese Text Classification Annotation Patterns - Feitosa, Alexander; Carneiro, Érica; Guedes, Gustavo (Anais do Symposium on Knowledge Discovery, Mining and Learning (KDMiLe), 2025)
This study explores how gendered annotation patterns influence sentiment classification in Brazilian Portuguese and whether these patterns are preserved—or amplified—by machine learning models. A corpus of 1,465 diary-style sentences was independently labeled by two gender-balanced annotator groups. Despite a high final agreement, as indicated by Cohen’s κ = 0.8177, statistical analyses revealed divergent annotation behaviors: male annotators exhibited higher internal consistency and lower entropy, while female annotators showed greater variability and a higher proportion of neutral labels. Classifiers trained separately on each group’s labels (SVM, LR, NB, RF, DT) reproduced these divergences to varying degrees. Notably, alignment between gendered models dropped to κ = 0.3838 (Decision Tree) and peaked at κ = 0.6952 (Logistic Regression), indicating that learning behaviors may differ substantially based on the annotation source. These findings reinforce that annotation is a socially based process. Gendered interpretive divergences can propagate through learning pipelines, shaping model behavior in ways that reflect and potentially amplify gender bias, often going unnoticed without annotation-aware evaluation strategies. Ethical approval was granted under protocol CAAE 82267824.8.0000.5289.