Beyond Systematic Bias: Investigating Gender Differences in Portuguese Text Classification Annotation Patterns - Feitosa, Alexander; Carneiro, Érica; Guedes, Gustavo (Anais do Symposium on Knowledge Discovery, Mining and Learning (KDMiLe); 2025: Anais do XIII Symposium on Knowledge Discovery, Mining and LearningDO - 10.5753/kdmile.2025.247589, 2025)
This study explores how gendered annotation patterns influence sentiment classification in Brazilian Portuguese and whether these patterns are preserved—or amplified—by machine learning models. A corpus of 1,465 diary-style sentences was independently labeled by two gender-balanced annotator groups. Despite a high final agreement, as indicated by Cohen’s κ = 0.8177, statistical analyses revealed divergent annotation behaviors: male annotators exhibited higher internal consistency and lower entropy, while female annotators showed greater variability and a higher proportion of neutral labels. Classifiers trained separately on each group’s labels (SVM, LR, NB, RF, DT) reproduced these divergences to varying degrees. Notably, alignment between gendered models dropped to κ = 0.3838 (Decision Tree) and peaked at κ = 0.6952 (Logistic Regression), indicating that learning behaviors may differ substantially based on the annotation source. These findings reinforce that annotation is a socially based process. Gendered interpretive divergences can propagate through learning pipelines, shaping model behavior in ways that reflect and potentially amplify gender bias, often going unnoticed without annotation-aware evaluation strategies. Ethical approval was granted under protocol CAAE 82267824.8.0000.5289.