Quando as Avaliações Viram Bombas: Explorando a Dinâmica do Review Bombing nos Jogos no Metacritic
Resumo
This study investigates the phenomenon of “Review bombing” on Metacritic, a platform for aggregating media reviews. Review bombs involve the deliberate posting of a large volume of negative reviews to alter the reputation of a media work, often driven by political, social, or creator-related disagreements. We collected extensive data from Metacritic, comprising 145K games and 1M reviews to understand and identify the characteristics of review bombing online. Using clustering techniques and analyzing various user and game-related metrics, we found that targeted games experience a significant increase in review volume, often with similar content and slightly higher toxicity, reflecting coordinated dissatisfaction campaigns. Additionally, the clustering achieved an F1-score of 0.5263, revealing particular challenges to categorizing this kind of attacks, such as the removal of evidence by platform moderation and difficulty in labeling minor attacks. The study highlights the need for robust mechanisms to detect and mitigate review bombs to preserve the integrity of review platforms and consumer trust.
Palavras-chave:
jogos, videogames, Metacritic, review bombing, clusterização, discurso de ódio, toxicidade
Referências
2022. O que é ’woke’ e por que o termo gera uma batalha cultural e política nos EUA. BBC. Online; Publicado em 8 de novembro de 2022. [link]
Nikolay Archak, Anindya Ghose, and Panagiotis G Ipeirotis. 2011. Deriving the pricing power of product features by mining consumer reviews. Management science 57, 8 (2011), 1485–1509.
Suman Biswas. 2023. Why is Baldur’s Gate 3 suddenly getting review bombed? SportsKeeda. [link]
Giulio Giacomo Cantone, Venera Tomaselli, and Valeria Mazzeo. 2021. Ideology-driven polarisation in online ratings: the review bombing of The Last of Us Part II. arXiv preprint arXiv:2104.01140 (2021).
Dalton Cooper. 2022. Diablo Immortal is Being Review Bombed. GAMERANT. Online; Publicado em 7 de Janeiro de 2022. [link]
Ryan Craddock. 2019. Metacritic Review-Bombers Strike Again With FIFA 20 On Switch. Nintendo Life. Online; Publicado em 30 de setembro de 2019. [link]
John Esposito. 2023. Diablo 4 players refuse to buy expansion for an “unfinished” game. Dexerto. Online; Publicado em 04 de novembro de 2023. [link]
Raphael Giannotti. 2023. Gamerant. JogadorescontinuamreclamandodoremasterdeGTATrilogy, masdessaveznoSteam Online; 20/01/2023. [link]
ADAMS GREENWOOD-ERICKSEN, SCOTT R POORMAN, and ROY PAPP. 2013. On the Validity of Metacritic in Assessing Game Value. Eludamos. Journal for Computer Game Culture 7, 1 (2013), 101–127.
Charlie Hall. 2019. Call of Duty: Modern Warfare review bombed over Russian portrayal. Polygon. Publicado em 28 de Outubro de 2019. [link]
Charlie Hall. 2019. Call of Duty: Modern Warfare’s Highway of Death controversy, explained. Polygon. [link]
Nan Hu, Ling Liu, and Jie Jennifer Zhang. 2008. Do online reviews affect product sales? The role of reviewer characteristics and temporal effects. Information Technology and management 9 (2008), 201–214.
Daniel Johnson, Christopher Watling, John Gardner, and Lennart E Nacke. 2014. The edge of glory: the relationship between metacritic scores and player experience. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play. 141–150.
Jordan K. Kobellarz and Thiago H. Silva. 2022. Should We Translate? Evaluating Toxicity in Online Comments when Translating from Portuguese to English. In Proceedings of the Brazilian Symposium on Multimedia and the Web (Curitiba,Brazil) (WebMedia ’22). ACM, 89–98. DOI: 10.1145/3539637.3556892
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004. Estimating mutual information. Physical review E 69, 6 (2004), 066138.
Ben Kuchera. 2017. The anatomy of a review bombing campaign. Polygon. Online; Publicado em 4 de Outubro de 2017. [link]
Lucas Lima, Julio CS Reis, Philipe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto. 2018. Inside the right-leaning echo chambers: Characterizing gab, an unmoderated social system. In 2018 ieee/acm international conference on advances in social networks analysis and mining (asonam). IEEE, 515–522.
Júlia Mazuco. 2018. DICE sobre Battlefield 5: personagens femininas vieram para ficar . Voxel. [link]
Philipe Melo and Clara Pimentel. 2022. A Campanha de Ódio contra The Last of Us Part II. In Anais Estendidos do XXI Simpósio Brasileiro de Jogos e Entretenimento Digital. SBC, 428–437.
Merriam-Webster.com Dictionary. 2024. Review bombing. Merriam-Webster. [link]
Rafael Monteiro. 2023. Diablo 4 tem bug que impede jogador de avançar no jogo; saiba evitar. Techtudo. Online; Publicado em 27 de junho de 2023. [link]
João Gabriel Nogueira. 2023. Review bombing: o que é, como reconhecer e Tem jeito de impedir? meuplaystation. Online; Publicado em 23 de junho de 2023. [link]
Tristan Ogilvie. 2021. Grand Theft Auto: The Trilogy - The Definitive Edition Review. IGN. Online; Publicado em 17 de novembro de 2021. [link]
Tom Phillips. 2011. Metacritic culls review-bombing users. EUROGAMER. Online; Publicado em 23 de Setembro de 2011. [link]
Johnny Reynolds. [n. d.]. Top 10 Times Video Games Got Review Bombed . watchmojo. [link]
José V. Rossi. 2023. Diablo 4 is Being Review Bombed on Steam . GAMERANT. Online; Publicado em 10 de Outubro de 2023. [link]
Gabriel Sales. 2023. Metacritic mudará estratégia de moderação após "review bomb"contra DLC de Horizon Forbidden West. IGN. Online; Publicado em 25 de Abril de 2023. [link]
Tiago Santos, Florian Lemmerich, Markus Strohmaier, and Denis Helic. 2019. What’s in a review: Discrepancies between expert and amateur reviews of video games on metacritic. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1–22.
Amanda Sharkey, Balázs Kovács, and Greta Hsu. 2023. Expert critics, rankings, and review aggregators: The changing nature of intermediation and the rise of markets with multiple intermediaries. Academy of Management Annals 17, 1 (2023), 1–36.
Bruna Telles. 2023. Another Diablo 4 Bug is Killing Players’ Progress. Gamerant. Online; Publicado em 14 de julho de 2023. [link]
Markos Viggiato, Dayi Lin, Abram Hindle, and Cor-Paul Bezemer. 2021. What causes wrong sentiment classifications of game reviews? IEEE Transactions on Games 14, 3 (2021), 350–363.
Asher Weiss. 2018. 21st century film criticism: the evolution of film criticism from professional intellectual analysis to a democratic phenomenon. (2018).
Wikipedia. 2024. List of review-bombing incidents. Wikipedia, The Free Encyclopedia.Online; Atualizado em 8 de maio de 2024. [link]
Preston Willke. 2021. 10 Games That Got Review-Bombed Before They Were Even Released . cbr. Online; Publicado em 25 de Agosto de 2021. [link]
Nikolay Archak, Anindya Ghose, and Panagiotis G Ipeirotis. 2011. Deriving the pricing power of product features by mining consumer reviews. Management science 57, 8 (2011), 1485–1509.
Suman Biswas. 2023. Why is Baldur’s Gate 3 suddenly getting review bombed? SportsKeeda. [link]
Giulio Giacomo Cantone, Venera Tomaselli, and Valeria Mazzeo. 2021. Ideology-driven polarisation in online ratings: the review bombing of The Last of Us Part II. arXiv preprint arXiv:2104.01140 (2021).
Dalton Cooper. 2022. Diablo Immortal is Being Review Bombed. GAMERANT. Online; Publicado em 7 de Janeiro de 2022. [link]
Ryan Craddock. 2019. Metacritic Review-Bombers Strike Again With FIFA 20 On Switch. Nintendo Life. Online; Publicado em 30 de setembro de 2019. [link]
John Esposito. 2023. Diablo 4 players refuse to buy expansion for an “unfinished” game. Dexerto. Online; Publicado em 04 de novembro de 2023. [link]
Raphael Giannotti. 2023. Gamerant. JogadorescontinuamreclamandodoremasterdeGTATrilogy, masdessaveznoSteam Online; 20/01/2023. [link]
ADAMS GREENWOOD-ERICKSEN, SCOTT R POORMAN, and ROY PAPP. 2013. On the Validity of Metacritic in Assessing Game Value. Eludamos. Journal for Computer Game Culture 7, 1 (2013), 101–127.
Charlie Hall. 2019. Call of Duty: Modern Warfare review bombed over Russian portrayal. Polygon. Publicado em 28 de Outubro de 2019. [link]
Charlie Hall. 2019. Call of Duty: Modern Warfare’s Highway of Death controversy, explained. Polygon. [link]
Nan Hu, Ling Liu, and Jie Jennifer Zhang. 2008. Do online reviews affect product sales? The role of reviewer characteristics and temporal effects. Information Technology and management 9 (2008), 201–214.
Daniel Johnson, Christopher Watling, John Gardner, and Lennart E Nacke. 2014. The edge of glory: the relationship between metacritic scores and player experience. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play. 141–150.
Jordan K. Kobellarz and Thiago H. Silva. 2022. Should We Translate? Evaluating Toxicity in Online Comments when Translating from Portuguese to English. In Proceedings of the Brazilian Symposium on Multimedia and the Web (Curitiba,Brazil) (WebMedia ’22). ACM, 89–98. DOI: 10.1145/3539637.3556892
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004. Estimating mutual information. Physical review E 69, 6 (2004), 066138.
Ben Kuchera. 2017. The anatomy of a review bombing campaign. Polygon. Online; Publicado em 4 de Outubro de 2017. [link]
Lucas Lima, Julio CS Reis, Philipe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto. 2018. Inside the right-leaning echo chambers: Characterizing gab, an unmoderated social system. In 2018 ieee/acm international conference on advances in social networks analysis and mining (asonam). IEEE, 515–522.
Júlia Mazuco. 2018. DICE sobre Battlefield 5: personagens femininas vieram para ficar . Voxel. [link]
Philipe Melo and Clara Pimentel. 2022. A Campanha de Ódio contra The Last of Us Part II. In Anais Estendidos do XXI Simpósio Brasileiro de Jogos e Entretenimento Digital. SBC, 428–437.
Merriam-Webster.com Dictionary. 2024. Review bombing. Merriam-Webster. [link]
Rafael Monteiro. 2023. Diablo 4 tem bug que impede jogador de avançar no jogo; saiba evitar. Techtudo. Online; Publicado em 27 de junho de 2023. [link]
João Gabriel Nogueira. 2023. Review bombing: o que é, como reconhecer e Tem jeito de impedir? meuplaystation. Online; Publicado em 23 de junho de 2023. [link]
Tristan Ogilvie. 2021. Grand Theft Auto: The Trilogy - The Definitive Edition Review. IGN. Online; Publicado em 17 de novembro de 2021. [link]
Tom Phillips. 2011. Metacritic culls review-bombing users. EUROGAMER. Online; Publicado em 23 de Setembro de 2011. [link]
Johnny Reynolds. [n. d.]. Top 10 Times Video Games Got Review Bombed . watchmojo. [link]
José V. Rossi. 2023. Diablo 4 is Being Review Bombed on Steam . GAMERANT. Online; Publicado em 10 de Outubro de 2023. [link]
Gabriel Sales. 2023. Metacritic mudará estratégia de moderação após "review bomb"contra DLC de Horizon Forbidden West. IGN. Online; Publicado em 25 de Abril de 2023. [link]
Tiago Santos, Florian Lemmerich, Markus Strohmaier, and Denis Helic. 2019. What’s in a review: Discrepancies between expert and amateur reviews of video games on metacritic. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1–22.
Amanda Sharkey, Balázs Kovács, and Greta Hsu. 2023. Expert critics, rankings, and review aggregators: The changing nature of intermediation and the rise of markets with multiple intermediaries. Academy of Management Annals 17, 1 (2023), 1–36.
Bruna Telles. 2023. Another Diablo 4 Bug is Killing Players’ Progress. Gamerant. Online; Publicado em 14 de julho de 2023. [link]
Markos Viggiato, Dayi Lin, Abram Hindle, and Cor-Paul Bezemer. 2021. What causes wrong sentiment classifications of game reviews? IEEE Transactions on Games 14, 3 (2021), 350–363.
Asher Weiss. 2018. 21st century film criticism: the evolution of film criticism from professional intellectual analysis to a democratic phenomenon. (2018).
Wikipedia. 2024. List of review-bombing incidents. Wikipedia, The Free Encyclopedia.Online; Atualizado em 8 de maio de 2024. [link]
Preston Willke. 2021. 10 Games That Got Review-Bombed Before They Were Even Released . cbr. Online; Publicado em 25 de Agosto de 2021. [link]
Publicado
14/10/2024
Como Citar
RIBEIRO, Marcus Vinicius Guerra; PIMENTEL, Clara Andrade; MELO, Philipe de Freitas.
Quando as Avaliações Viram Bombas: Explorando a Dinâmica do Review Bombing nos Jogos no Metacritic. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 30. , 2024, Juiz de Fora/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 249-256.
DOI: https://doi.org/10.5753/webmedia.2024.241744.
