Biases in GPT-3.5 Turbo model: a case study regarding gender and language - Assi, Fernanda Malheiros; Caseli, Helena de Medeiros (Anais do Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), 2024)
Interactions with Generative Language Models like OpenAI’s GPT3.5 Turbo are increasingly common in everyday life, making it essential to examine their potential biases. This study assesses biases in the GPT-3.5 Turbo model using the regard metric, which evaluates the level of respect or esteem expressed towards different demographic groups. Specifically, we investigate how the model perceives regard towards different genders (male, female, and neutral) in both English and Portuguese. To achieve this, we isolated three variables (gender, language, and moderation filters) and analyzed their individual impacts on the model’s outputs. Our results indicate a slight positive bias towards feminine over masculine and neutral genders, a more favorable bias towards English compared to Portuguese, and consistently more negative outputs when we attempted to reduce the moderation filters.