Identifying Confirmation Bias in a Search as Learning Task: A Study on The Use of Artificial Intelligence in Education
Resumo
Confirmation bias, the tendency to favor information that supports existing beliefs, can hinder information-seeking, especially in learning contexts where it can perpetuate a one-sided perspective. This paper examines how confirmation bias affects search behaviors among 84 participants learning about AI in education. Participants were divided into Neutral and Biased groups based on their prior attitudes, with the Biased group receiving reinforcing information beforehand. Participants’ interactions with the search system were logged, and we analyzed the data for behavioral differences. Results showed that biased participants often completed searches quickly, spending less time engaging with and selecting search results, and issued longer queries. However, other variables showed no statistical difference. Some results contradict other studies on confirmation bias in search, highlighting the complexity of search dynamics in learning contexts and suggesting the need for specialized research into cognitive biases in search as a learning process.
Referências
Athukorala, K., Głowacka, D., Jacucci, G., Oulasvirta, A., and Vreeken, J. (2016). Is exploratory search different? a comparison of information search behavior for exploratory and lookup tasks. Journal of the Association for Information Science and Technology, 67(11):2635–2651.
Machado, M., Assis, E. C., Souza, J. F., and Siqueira, S. W. M. (2024). A framework to support experimentation in the context of cognitive biases in search as a learning process. In Proceedings of the 20th Brazilian Symposium on Information Systems, SBSI '24, New York, NY, USA. Association for Computing Machinery.
Machado, M., Gimenez, P., and Siqueira, S. (2020). Raising the dimensions and variables for searching as a learning process: A systematic mapping of the literature. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 1393–1402, Porto Alegre, RS, Brasil. SBC.
Gimenez, P., Machado, M., Pinelli, C., and Siqueira, S. (2020). Investigating the learning perspective of searching as learning, a review of the state of the art. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 302–311, Porto Alegre, RS, Brasil. SBC.
Suzuki, M. and Yamamoto, Y. (2021). Analysis of relationship between confirmation bias and web search behavior. In Proceedings of the 22nd International Conference on Information Integration and Web-Based Applications & Services, iiWAS '20, page 184–191, New York, NY, USA. Association for Computing Machinery.
Azzopardi, L. (2021). Cognitive biases in search: A review and reflection of cognitive biases in information retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, CHIIR '21, page 27–37, New York, NY, USA. Association for Computing Machinery.
Baeza-Yates, R. (2018). Bias on the web. Communications of the ACM, 61(6):54–61.
Bateman, S., Teevan, J., and White, R. W. (2012). The search dashboard: how reflection and comparison impact search behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1785–1794.
Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian journal of information science, 5(1):133–143
Benson, B. (2016). Cognitive bias cheat sheet. [link]. [Online; accessed July-2023].
Festinger, L. (1957). A theory of cognitive dissonance, volume 2. Stanford university press.
Gagne, R. M. (1968). Contributions of learning to human development. Psychological review, 75(3):177.
Gomroki, G., Behzadi, H., Fattahi, R., and Salehi Fadardi, J. (2023). Identifying effective cognitive biases in information retrieval. Journal of Information Science, 49(2):348–358.
Haselton, M. G., Nettle, D., and Murray, D. R. (2015). The evolution of cognitive bias. In The Handbook of Evolutionary Psychology, pages 1–20, Online. John Wiley & Sons, Ltd.
Hirschman, E. C. (1983). Judgement under uncertainty: Heuristics and biases. JMR, Journal of Marketing Research (pre-1986), 20(000002):217.
Kelly, D. and Sugimoto, C. R. (2013). A systematic review of interactive information retrieval evaluation studies, 1967–2006. Journal of the American Society for Information Science and Technology, 64(4):745–770.
Kong, H.-K., Liu, Z., and Karahalios, K. (2019). Trust and recall of information across varying degrees of title-visualization misalignment. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–13.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology, 2(2):175–220.
Otto, C., Rokicki, M., Pardi, G., Gritz, W., Hienert, D., Yu, R., von Hoyer, J., Hoppe, A., Dietze, S., Holtz, P., et al. (2022). Sal-lightning dataset: Search and eye gaze behavior, resource interactions and knowledge gain during web search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, pages 347–352.
Rieger, A., Draws, T., Theune, M., and Tintarev, N. (2021). This item might reinforce your opinion: Obfuscation and labeling of search results to mitigate confirmation bias. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media, HT ’21, page 189–199, New York, NY, USA. Association for Computing Machinery.
Rieh, S. Y., Collins-Thompson, K., Hansen, P., and Lee, H.-J. (2016). Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science, 42(1):19–34.
Soufan, A., Ruthven, I., and Azzopardi, L. (2022). Searching the literature: an analysis of an exploratory search task. In ACM SIGIR Conference on Human Information Interaction and Retrieval, pages 146–157.
Tversky, A. and Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185(4157):1124–1131.
Vakkari, P. (2016). Searching as learning: A systematization based on literature. Journal of Information Science, 42(1):7–18.
White, R. W. and Morris, D. (2007). Investigating the querying and browsing behavior of advanced search engine users. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 255–262.
Machado, M., Assis, E. C., Souza, J. F., and Siqueira, S. W. M. (2024). A framework to support experimentation in the context of cognitive biases in search as a learning process. In Proceedings of the 20th Brazilian Symposium on Information Systems, SBSI '24, New York, NY, USA. Association for Computing Machinery.
Machado, M., Gimenez, P., and Siqueira, S. (2020). Raising the dimensions and variables for searching as a learning process: A systematic mapping of the literature. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 1393–1402, Porto Alegre, RS, Brasil. SBC.
Gimenez, P., Machado, M., Pinelli, C., and Siqueira, S. (2020). Investigating the learning perspective of searching as learning, a review of the state of the art. In Anais do XXXI Simpósio Brasileiro de Informática na Educação, pages 302–311, Porto Alegre, RS, Brasil. SBC.
Suzuki, M. and Yamamoto, Y. (2021). Analysis of relationship between confirmation bias and web search behavior. In Proceedings of the 22nd International Conference on Information Integration and Web-Based Applications & Services, iiWAS '20, page 184–191, New York, NY, USA. Association for Computing Machinery.
Azzopardi, L. (2021). Cognitive biases in search: A review and reflection of cognitive biases in information retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, CHIIR '21, page 27–37, New York, NY, USA. Association for Computing Machinery.
Baeza-Yates, R. (2018). Bias on the web. Communications of the ACM, 61(6):54–61.
Bateman, S., Teevan, J., and White, R. W. (2012). The search dashboard: how reflection and comparison impact search behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1785–1794.
Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian journal of information science, 5(1):133–143
Benson, B. (2016). Cognitive bias cheat sheet. [link]. [Online; accessed July-2023].
Festinger, L. (1957). A theory of cognitive dissonance, volume 2. Stanford university press.
Gagne, R. M. (1968). Contributions of learning to human development. Psychological review, 75(3):177.
Gomroki, G., Behzadi, H., Fattahi, R., and Salehi Fadardi, J. (2023). Identifying effective cognitive biases in information retrieval. Journal of Information Science, 49(2):348–358.
Haselton, M. G., Nettle, D., and Murray, D. R. (2015). The evolution of cognitive bias. In The Handbook of Evolutionary Psychology, pages 1–20, Online. John Wiley & Sons, Ltd.
Hirschman, E. C. (1983). Judgement under uncertainty: Heuristics and biases. JMR, Journal of Marketing Research (pre-1986), 20(000002):217.
Kelly, D. and Sugimoto, C. R. (2013). A systematic review of interactive information retrieval evaluation studies, 1967–2006. Journal of the American Society for Information Science and Technology, 64(4):745–770.
Kong, H.-K., Liu, Z., and Karahalios, K. (2019). Trust and recall of information across varying degrees of title-visualization misalignment. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–13.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology, 2(2):175–220.
Otto, C., Rokicki, M., Pardi, G., Gritz, W., Hienert, D., Yu, R., von Hoyer, J., Hoppe, A., Dietze, S., Holtz, P., et al. (2022). Sal-lightning dataset: Search and eye gaze behavior, resource interactions and knowledge gain during web search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, pages 347–352.
Rieger, A., Draws, T., Theune, M., and Tintarev, N. (2021). This item might reinforce your opinion: Obfuscation and labeling of search results to mitigate confirmation bias. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media, HT ’21, page 189–199, New York, NY, USA. Association for Computing Machinery.
Rieh, S. Y., Collins-Thompson, K., Hansen, P., and Lee, H.-J. (2016). Towards searching as a learning process: A review of current perspectives and future directions. Journal of Information Science, 42(1):19–34.
Soufan, A., Ruthven, I., and Azzopardi, L. (2022). Searching the literature: an analysis of an exploratory search task. In ACM SIGIR Conference on Human Information Interaction and Retrieval, pages 146–157.
Tversky, A. and Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185(4157):1124–1131.
Vakkari, P. (2016). Searching as learning: A systematization based on literature. Journal of Information Science, 42(1):7–18.
White, R. W. and Morris, D. (2007). Investigating the querying and browsing behavior of advanced search engine users. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 255–262.
Publicado
04/11/2024
Como Citar
MACHADO, Marcelo; SOUZA, Jairo F. de; SIQUEIRA, Sean W. M..
Identifying Confirmation Bias in a Search as Learning Task: A Study on The Use of Artificial Intelligence in Education. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 35. , 2024, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 1208-1221.
DOI: https://doi.org/10.5753/sbie.2024.242341.
