An exploratory research about ethical issues on a smart toy: The Hello Barbie case study
Resumo
Smart toys are becoming increasingly present in children's lives, reinforcing the relevance of this market niche. Advances in user interfaces and artificial intelligence have been incorporated into smart toys to provide greater autonomy and inductive reasoning skills through machine learning. However, machine learning embedded in smart toys not only brings benefits but also potential problems of bias, possibly related to prejudice and discrimination. This work aims to explore Mattel's Hello Barbie smart toy in a case study, seeking to analyze its knowledge base and conversational functionality to identify possible ethical issues that could cause harm to children. The intention is to show unknown risks that can occur in the evolution's process of smart toys.
Palavras-chave:
smart toys, hello barbie, machine learning, bias, prejudice
Referências
Albuquerque, O. d. P., Fantinato, M., Eler, M. M., Peres, S. M., and Hung, P. C. K. (2020). A study of parental control requirements for smart toys. In IEEE International Conference on Systems, Man, and Cybernetics. submitted.
Albuquerque, O. d. P., Fantinato, M., Hung, P. C., Peres, S. M., Iqbal, F., Rehman, U., and Shah, M. U. (2022). Recommendations for a smart toy parental control tool. The Journal of Supercomputing, pages 1–39.
Albuquerque, O. d. P., Fantinato, M., Kelner, J., and Albuquerque, A. P. (2019). Privacy in smart toys: Risks and proposed solutions. Electronic Commerce Research and Applications, 39:1–15.
Angwin, J., Larson, J., M. S., and Kirchner, L. (2016). Machine bias. http://tiny.cc/sazdaz. ProPublica.
Bird, S., Kenthapadi, K., Kiciman, E., and Mitchell, M. (2019). Fairness-aware machine learning: Practical challenges and lessons learned. In 12th International Conference on Web Search and Data Mining, pages 834–835.
Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V., and Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29.
de Oliveira-Menegotto, L. M., Pasini, A. I., and Levandowski, G. (2013). O bullying escolar no brasil: uma revisão de artigos científicos. Psicologia: Teoria e Prática, 15(2):203–215. (in Portuguese).
Fantinato, M., Albuquerque, O. D. P., De Albuquerque, A. P., Kelner, J., and Yankson, B. (2020). A literature survey on smart toy-related children’s privacy risks. In 53rd Hawaii International Conference on System Sciences.
Fantinato, M., Hung, P. C., Jiang, Y., Roa, J., Villarreal, P., Melaisi, M., and Amancio, F. (2017). A survey on purchase intention of hello barbie in brazil and argentina. In Computing in Smart Toys, pages 21–34. Springer.
Fantinato, M., Hung, P. C. K., Jiang, Y., Roa, J., Villarreal, P., Melaisi, M., and Amancio, F. (2018). A preliminary study of Hello Barbie in Brazil and Argentina. Sustainable Cities and Society, 40:83–90.
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., and Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. In Conference on Fairness, Account., and Transp., pages 329–338.
Gil, A. C. (2002). Como elaborar projetos de pesquisa. São Paulo, 5(61):16–17. (in Portuguese).
Hajian, S., Bonchi, F., and Castillo, C. (2016). Algorithmic bias: From discrimination discovery to fairness-aware data mining. In International Conference on Knowl. Disc. and Data Mining, volume 13-17-Aug, pages 2125–2126.
Hajian, S. and Domingo-Ferrer, J. (2013). A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and Data Engineering, 25(7):1445–1459.
Harris, S. (2015). ‘Hell No Barbie’ campaign targets Hello Barbie over privacy concerns. http://tiny.cc/pbzdaz. CBC.
Hern, A. (2020). Twitter apologises for “racist” image-cropping algorithm. [link].
Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudik, M., and Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need? In Conference on Hum. Fact. in Comp. Sys., page 600.
Hung, P. C. K., Fantinato, M., and Rafferty, L. (2016a). A study of privacy requirements for smart toys. In 20th Pacific Asia Conference on Information Systems, pages 1–7.
Hung, P. C. K., Iqbal, F., Huang, S.-C., Melaisi, M., and Pang, K. (2016b). A glance of child’s play privacy in smart toys. In 2nd International Conference on Cloud Computing and Security, pages 217–231.
Jupiter Research (2018). Smart toy revenues to grow by almost 200% from 2018 to $18 billion by 2023. [link].
Liu, J. and Graves, N. (2011). Childhood bullying: A review of constructs, concepts, and nursing implications. Public Health Nursing, 28(6):556–568.
Mahmoud, M., Hossen, M. Z., Barakat, H., Mannan, M., and Youssef, A. (2017). Towards a comprehensive analytical framework for smart toy privacy practices. In 7thWorkshop on Socio-Technical Aspects in Security and Trust, pages 64–75.
Manzini, T., Lim, Y. C., Tsvetkov, Y., and Black, A. W. (2019). Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings. arXiv preprint arXiv:1904.04047.
Mathews, L. (2017). The latest privacy nightmare for parents: Data leaks from smart toys. http://bit.do/e3cCY. Forbes.
Mattel’s Hello Barbie (2017). Frequently asked questions (FAQ). http://hellobarbiefaq.mattel.com/.
Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., and Scheidt, P. (2001). Bullying behaviors among us youth: Prevalence and association with psychosocial adjustment. JAMA, 285(16):2094–2100.
Pedreshi, D., Ruggieri, S., and Turini, F. (2008). Discrimination-aware data mining. In International Conference on Knowledge Discovery and Data Mining, pages 560–568.
Powlishta, K. K., Serbin, L. A., Doyle, A.-B., and White, D. R. (1994). Gender, ethnic, and body type biases: The generality of prejudice in childhood. Developmental Psychology, 30(4):526.
Rafferty, L., Fantinato, M., and Hung, P. C. K. (2015). Privacy requirements in toy computing. In Hung, P., editor, Mobile Services for Toy Computing, pages 141–173. Springer.
Rafferty, L., Hung, P., Fantinato, M., Peres, S. M., Iqbal, F., Kuo, S., and Huang, S. (2017). Towards a privacy rule conceptual model for smart toys. In 50th Hawaii International Conference on System Sciences, pages 1–10.
Ruggieri, S., Hajian, S., Kamiran, F., and Zhang, X. (2014). Anti-discrimination analysis using privacy attack strategies. In Joint Eur. Conference on Mach. Learn. and Knowl. Disc. in Datab., pages 694–710.
Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc.
Tang, J. K. and Hung, P. C. (2017). Computing in Smart Toys. Springer.
Wang, J., Iannotti, R. J., Luk, J. W., and Nansel, T. R. (2010). Co-occurrence of victimization from five subtypes of bullying: Physical, verbal, social exclusion, spreading rumors, and cyber. Journal of Pediatric Psychology, 35(10):1103–1112.
Wang, K., Wang, P., Fu, A. W., and Wong, R. C.-W. (2016). Generalized bucketization scheme for flexible privacy settings. Inform. Sci., 348:377 – 393.
Albuquerque, O. d. P., Fantinato, M., Hung, P. C., Peres, S. M., Iqbal, F., Rehman, U., and Shah, M. U. (2022). Recommendations for a smart toy parental control tool. The Journal of Supercomputing, pages 1–39.
Albuquerque, O. d. P., Fantinato, M., Kelner, J., and Albuquerque, A. P. (2019). Privacy in smart toys: Risks and proposed solutions. Electronic Commerce Research and Applications, 39:1–15.
Angwin, J., Larson, J., M. S., and Kirchner, L. (2016). Machine bias. http://tiny.cc/sazdaz. ProPublica.
Bird, S., Kenthapadi, K., Kiciman, E., and Mitchell, M. (2019). Fairness-aware machine learning: Practical challenges and lessons learned. In 12th International Conference on Web Search and Data Mining, pages 834–835.
Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V., and Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29.
de Oliveira-Menegotto, L. M., Pasini, A. I., and Levandowski, G. (2013). O bullying escolar no brasil: uma revisão de artigos científicos. Psicologia: Teoria e Prática, 15(2):203–215. (in Portuguese).
Fantinato, M., Albuquerque, O. D. P., De Albuquerque, A. P., Kelner, J., and Yankson, B. (2020). A literature survey on smart toy-related children’s privacy risks. In 53rd Hawaii International Conference on System Sciences.
Fantinato, M., Hung, P. C., Jiang, Y., Roa, J., Villarreal, P., Melaisi, M., and Amancio, F. (2017). A survey on purchase intention of hello barbie in brazil and argentina. In Computing in Smart Toys, pages 21–34. Springer.
Fantinato, M., Hung, P. C. K., Jiang, Y., Roa, J., Villarreal, P., Melaisi, M., and Amancio, F. (2018). A preliminary study of Hello Barbie in Brazil and Argentina. Sustainable Cities and Society, 40:83–90.
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., and Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. In Conference on Fairness, Account., and Transp., pages 329–338.
Gil, A. C. (2002). Como elaborar projetos de pesquisa. São Paulo, 5(61):16–17. (in Portuguese).
Hajian, S., Bonchi, F., and Castillo, C. (2016). Algorithmic bias: From discrimination discovery to fairness-aware data mining. In International Conference on Knowl. Disc. and Data Mining, volume 13-17-Aug, pages 2125–2126.
Hajian, S. and Domingo-Ferrer, J. (2013). A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and Data Engineering, 25(7):1445–1459.
Harris, S. (2015). ‘Hell No Barbie’ campaign targets Hello Barbie over privacy concerns. http://tiny.cc/pbzdaz. CBC.
Hern, A. (2020). Twitter apologises for “racist” image-cropping algorithm. [link].
Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudik, M., and Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need? In Conference on Hum. Fact. in Comp. Sys., page 600.
Hung, P. C. K., Fantinato, M., and Rafferty, L. (2016a). A study of privacy requirements for smart toys. In 20th Pacific Asia Conference on Information Systems, pages 1–7.
Hung, P. C. K., Iqbal, F., Huang, S.-C., Melaisi, M., and Pang, K. (2016b). A glance of child’s play privacy in smart toys. In 2nd International Conference on Cloud Computing and Security, pages 217–231.
Jupiter Research (2018). Smart toy revenues to grow by almost 200% from 2018 to $18 billion by 2023. [link].
Liu, J. and Graves, N. (2011). Childhood bullying: A review of constructs, concepts, and nursing implications. Public Health Nursing, 28(6):556–568.
Mahmoud, M., Hossen, M. Z., Barakat, H., Mannan, M., and Youssef, A. (2017). Towards a comprehensive analytical framework for smart toy privacy practices. In 7thWorkshop on Socio-Technical Aspects in Security and Trust, pages 64–75.
Manzini, T., Lim, Y. C., Tsvetkov, Y., and Black, A. W. (2019). Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings. arXiv preprint arXiv:1904.04047.
Mathews, L. (2017). The latest privacy nightmare for parents: Data leaks from smart toys. http://bit.do/e3cCY. Forbes.
Mattel’s Hello Barbie (2017). Frequently asked questions (FAQ). http://hellobarbiefaq.mattel.com/.
Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., and Scheidt, P. (2001). Bullying behaviors among us youth: Prevalence and association with psychosocial adjustment. JAMA, 285(16):2094–2100.
Pedreshi, D., Ruggieri, S., and Turini, F. (2008). Discrimination-aware data mining. In International Conference on Knowledge Discovery and Data Mining, pages 560–568.
Powlishta, K. K., Serbin, L. A., Doyle, A.-B., and White, D. R. (1994). Gender, ethnic, and body type biases: The generality of prejudice in childhood. Developmental Psychology, 30(4):526.
Rafferty, L., Fantinato, M., and Hung, P. C. K. (2015). Privacy requirements in toy computing. In Hung, P., editor, Mobile Services for Toy Computing, pages 141–173. Springer.
Rafferty, L., Hung, P., Fantinato, M., Peres, S. M., Iqbal, F., Kuo, S., and Huang, S. (2017). Towards a privacy rule conceptual model for smart toys. In 50th Hawaii International Conference on System Sciences, pages 1–10.
Ruggieri, S., Hajian, S., Kamiran, F., and Zhang, X. (2014). Anti-discrimination analysis using privacy attack strategies. In Joint Eur. Conference on Mach. Learn. and Knowl. Disc. in Datab., pages 694–710.
Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc.
Tang, J. K. and Hung, P. C. (2017). Computing in Smart Toys. Springer.
Wang, J., Iannotti, R. J., Luk, J. W., and Nansel, T. R. (2010). Co-occurrence of victimization from five subtypes of bullying: Physical, verbal, social exclusion, spreading rumors, and cyber. Journal of Pediatric Psychology, 35(10):1103–1112.
Wang, K., Wang, P., Fu, A. W., and Wong, R. C.-W. (2016). Generalized bucketization scheme for flexible privacy settings. Inform. Sci., 348:377 – 393.
Publicado
31/07/2022
Como Citar
ALBUQUERQUE, Otávio de Paula; FANTINATO, Marcelo; PERES, Sarajane Marques; HUNG, Patrick C. K..
An exploratory research about ethical issues on a smart toy: The Hello Barbie case study. In: WORKSHOP SOBRE AS IMPLICAÇÕES DA COMPUTAÇÃO NA SOCIEDADE (WICS), 3. , 2022, Niterói.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 67-75.
ISSN 2763-8707.
DOI: https://doi.org/10.5753/wics.2022.222803.
