Applying the engagement by design methodology: perceptions and lessons learned
Resumo
Crowdsourcing initiatives have been applied in a wide range of domains. In crowdsourcing applications where user-generated data is the basis of the application, sporadic usage is not enough, users must engage with it. To ensure, from the design stage, that a solution has strong engagement, we developed an 18-card deck as part of a design thinking approach that provides domain-specific insights for designers in crowdsourcing initiatives. We applied the cards in a study where we analyzed five online design workshops and reflected on the results from the perspective of 14 facilitators and 17 workshop participants. The results led to a better understanding of the methodology and suggested changes and customizations of the playbook instructions.
Palavras-chave:
design, crowdsourcing, engagement
Referências
Simon à Campo, Vasssilis-Javed Khan, Konstantinos Papangelis, and Panos Markopoulos. 2019. Community heuristics for user interface evaluation of crowd-sourcing platforms. Future Generation Computer Systems 95 (2019), 775--789.
Simon Attfield, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. Towards a science of user engagement (position paper). In WSDM workshop on user modelling for Web applications. 9--12.
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.
Richard Cave, Karina Kocemba, Svjetlana Dajic, Holloe Bostock, and Alastair Cook. 2019. PALS: Patching ALS through Crowdsourced Advice, Social Links & Public Awareness. In Extended Abstracts of the 2019 CHI. 1--6.
Lucas Colusso, Cynthia L Bennett, Gary Hsieh, and Sean A Munson. 2017. Translational resources: Reducing the gap between academic research and HCI practice. In DIS. 957--968.
Lucas Colusso, Tien Do, and Gary Hsieh. 2018. Behavior change design sprints. In Designing Interactive Systems Conference. 791--803.
Mônica da Silva, José Viterbo, Flavia Bernardini, and Cristiano Maciel. 2018. Identifying Privacy Functional Requirements for Crowdsourcing Applications in Smart Cities. In 2018 IEEE ISI. IEEE, 106--111.
Ying Deng, Alissa N Antle, and Carman Neustaedter. 2014. Tango cards: a card-based design tool for informing the design of tangible learning games. In Proceedings of the 2014 conference on Designing interactive systems. 695--704.
Dejan Drajic, Karl Andersson, Kai Zhang, Nathalie Stembert, Katariina Malmberg, Anna Brékine, Wim Vanobberghen, Abdolrasoul Habibipour, and Jan Waeben. 2019. User Engagement for Large Scale Pilots in the Internet of Things. In TELSIKS. IEEE, 46--53.
Nir Eyal. 2014. Hooked: How to build habit-forming products. Penguin.
Anton Fedosov, Masako Kitazaki, William Odom, and Marc Langheinrich. 2019. Sharing economy design cards. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--14.
Ulrike Felt, Simone Schumann, Claudia G Schwarz, and Michael Strassnig. 2014. Technology of imagination: a card-based public engagement method for debating emerging technologies. Qualitative research 14, 2 (2014), 233--251.
Gabriela Goldschmidt. 2016. Linkographic evidence for concurrent divergent and convergent thinking in creative design. Creativity research journal 28, 2 (2016), 115--122.
Zhuojun Gu, Ravi Bapna, Jason Chan, and Alok Gupta. 2022. Measuring the impact of crowdsourcing features on mobile app user engagement and retention: A randomized field experiment. Management Science 68, 2 (2022), 1297--1329.
Ido Guy, Anat Hashavit, and Yaniv Corem. 2015. Games for crowds: A crowdsourcing game platform for the enterprise. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1860--1871.
Kim Halskov and Peter Dalsgård. 2006. Inspiration card workshops. In Proceedings of the Designing Interactive systems. 2--11.
Michael Brandon Haworth, Muhammad Usman, Davide Schaumann, Nilay Chakraborty, Glen Berseth, Petros Faloutsos, and Mubbasir Kapadia. 2020. Gamification of Crowd-Driven Environment Design. IEEE Computer Graphics and Applications (2020).
Sara Hultqvist, Oskar Jonsson, Håkan Jönson, and Susanne Iwarsson. 2021. Collaboration in grant proposals and assessments in ageing research-justification or a quest for a collaborology? Social Epistemology 35, 5 (2021), 427--440.
Elizabeth Johnson and Chern Li Liew. 2020. Engagement-oriented design: a study of New Zealand public cultural heritage institutions crowdsourcing platforms. Online Information Review (2020).
Aikaterini Katmada, Anna Satsiou, and Ioannis Kompatsiaris. 2016. Incentive mechanisms for crowdsourcing platforms. In International Conference on Internet Science. Springer, 3--18.
Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. 2014. Measuring user engagement. Synthesis Lectures on Information Concepts, Retrieval, and Services 6, 4 (2014), 1--132.
Ju Hyun Lee and Michael J Ostwald. 2022. The relationship between divergent thinking and ideation in the conceptual design process. Design Studies 79 (2022), 101089.
Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In International conference on user modeling, adaptation, and personalization. Springer, 164--175.
Sophia B Liu. 2014. Crisis crowdsourcing framework: Designing strategic configurations of crowdsourcing for the emergency management domain. CSCW 23, 4--6 (2014), 389--443.
Nick Logler, Daisy Yoo, and Batya Friedman. 2018. Metaphor cards: A how-to-guide for making and using a generative metaphorical design toolkit. In Proceedings of the 2018 Designing Interactive Systems Conference. 1373--1386.
James Derek Lomas, Mihovil Karac, and Mathieu Gielen. 2021. Design Space Cards: Using a Card Deck to Navigate the Design Space of Interactive Play. Proceedings of the ACM on Human-Computer Interaction 5, CHIPLAY (2021), 227.
Andrés Lucero, Peter Dalsgaard, Kim Halskov, and Jacob Buur. 2016. Designing with cards. In Collaboration in creative design. Springer, 75--95.
Ewa Luger, Lachlan Urquhart, Tom Rodden, and Michael Golembewski. 2015. Playing the legal card: Using ideation cards to raise data protection issues within the design process. In Proceedings of the 33rd Annual ACM conference on human factors in computing systems. 457--466.
Mateus Monteiro, Leonardo Vasconcelos, José Viterbo, Luciana Salgado, and Flávia Bernardini. 2021. Assessing the Quality of Local E-Government Service Through Citizen-Sourcing Applications. In 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD). 1178--1183.
Simone Mora, Francesco Gianni, and Monica Divitini. 2017. Tiles: a card-based ideation toolkit for the internet of things. In Proceedings of the 2017 conference on designing interactive systems. 587--598.
Iohanna Nicenboim, Elisa Giaccardi, and Lenneke Kuijer. 2018. Designing connected resources for older people. In Proceedings of the 2018 Designing Interactive Systems Conference. 413--425.
Donald A Norman. 1999. Affordance, conventions, and design. interactions 6, 3 (1999), 38--43.
Heather L O'Brien and Elaine G Toms. 2010. The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology 61, 1 (2010), 50--69.
Heather L O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112 (2018), 28--39.
Sangkeun Park, Sujin Kwon, and Uichin Lee. 2018. Campuswatch: Exploring community sourced patrolling with pervasive mobile technology. PACM HCI 2, CSCW (2018), 1--25.
David J Roedl and Erik Stolterman. 2013. Design research at CHI and its applicability to design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1951--1954.
Robin Roy and James P Warren. 2019. Card-based design tools: A review and analysis of 155 card decks for designers and designing. Design Studies 63 (2019), 125--154.
Lei Shi, James MacKrill, Elisavet Dimitrokali, and Rebecca Cain. 2015. Digital Crowdsourcing in Healthcare Environment Co-design. (2015).
Rabail Tahir and Alf Inge Wang. 2020. Transforming a theoretical framework to design cards: LEAGUE ideation toolkit for game-based learning design. Sustainability 12, 20 (2020), 8487.
Ramine Tinati, Max Van Kleek, Elena Simperl, Markus Luczak-Rösch, Robert Simpson, and Nigel Shadbolt. 2015. Designing for citizen data analysis: A cross-sectional case study of a multi-domain citizen science platform. In ACM Conference on Human Factors in Computing Systems. 4069--4078.
Leonardo Vasconcelos, Raissa Barcellos, José Viterbo, Flavia Bernardini, Clodis Boscarioli, and Eunice Nunes. 2019. How factors that influence engagement impact users' evaluations in mobile app stores. In International Conference on Human-Computer Interaction. Springer, 571--584.
Leonardo Vasconcelos, Daniela Trevisan, and José Viterbo. 2022. Engagement by Design: A Card-based approach to design crowdsourcing initiatives. In 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 353--358.
Simon Attfield, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. Towards a science of user engagement (position paper). In WSDM workshop on user modelling for Web applications. 9--12.
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77--101.
Richard Cave, Karina Kocemba, Svjetlana Dajic, Holloe Bostock, and Alastair Cook. 2019. PALS: Patching ALS through Crowdsourced Advice, Social Links & Public Awareness. In Extended Abstracts of the 2019 CHI. 1--6.
Lucas Colusso, Cynthia L Bennett, Gary Hsieh, and Sean A Munson. 2017. Translational resources: Reducing the gap between academic research and HCI practice. In DIS. 957--968.
Lucas Colusso, Tien Do, and Gary Hsieh. 2018. Behavior change design sprints. In Designing Interactive Systems Conference. 791--803.
Mônica da Silva, José Viterbo, Flavia Bernardini, and Cristiano Maciel. 2018. Identifying Privacy Functional Requirements for Crowdsourcing Applications in Smart Cities. In 2018 IEEE ISI. IEEE, 106--111.
Ying Deng, Alissa N Antle, and Carman Neustaedter. 2014. Tango cards: a card-based design tool for informing the design of tangible learning games. In Proceedings of the 2014 conference on Designing interactive systems. 695--704.
Dejan Drajic, Karl Andersson, Kai Zhang, Nathalie Stembert, Katariina Malmberg, Anna Brékine, Wim Vanobberghen, Abdolrasoul Habibipour, and Jan Waeben. 2019. User Engagement for Large Scale Pilots in the Internet of Things. In TELSIKS. IEEE, 46--53.
Nir Eyal. 2014. Hooked: How to build habit-forming products. Penguin.
Anton Fedosov, Masako Kitazaki, William Odom, and Marc Langheinrich. 2019. Sharing economy design cards. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--14.
Ulrike Felt, Simone Schumann, Claudia G Schwarz, and Michael Strassnig. 2014. Technology of imagination: a card-based public engagement method for debating emerging technologies. Qualitative research 14, 2 (2014), 233--251.
Gabriela Goldschmidt. 2016. Linkographic evidence for concurrent divergent and convergent thinking in creative design. Creativity research journal 28, 2 (2016), 115--122.
Zhuojun Gu, Ravi Bapna, Jason Chan, and Alok Gupta. 2022. Measuring the impact of crowdsourcing features on mobile app user engagement and retention: A randomized field experiment. Management Science 68, 2 (2022), 1297--1329.
Ido Guy, Anat Hashavit, and Yaniv Corem. 2015. Games for crowds: A crowdsourcing game platform for the enterprise. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1860--1871.
Kim Halskov and Peter Dalsgård. 2006. Inspiration card workshops. In Proceedings of the Designing Interactive systems. 2--11.
Michael Brandon Haworth, Muhammad Usman, Davide Schaumann, Nilay Chakraborty, Glen Berseth, Petros Faloutsos, and Mubbasir Kapadia. 2020. Gamification of Crowd-Driven Environment Design. IEEE Computer Graphics and Applications (2020).
Sara Hultqvist, Oskar Jonsson, Håkan Jönson, and Susanne Iwarsson. 2021. Collaboration in grant proposals and assessments in ageing research-justification or a quest for a collaborology? Social Epistemology 35, 5 (2021), 427--440.
Elizabeth Johnson and Chern Li Liew. 2020. Engagement-oriented design: a study of New Zealand public cultural heritage institutions crowdsourcing platforms. Online Information Review (2020).
Aikaterini Katmada, Anna Satsiou, and Ioannis Kompatsiaris. 2016. Incentive mechanisms for crowdsourcing platforms. In International Conference on Internet Science. Springer, 3--18.
Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. 2014. Measuring user engagement. Synthesis Lectures on Information Concepts, Retrieval, and Services 6, 4 (2014), 1--132.
Ju Hyun Lee and Michael J Ostwald. 2022. The relationship between divergent thinking and ideation in the conceptual design process. Design Studies 79 (2022), 101089.
Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In International conference on user modeling, adaptation, and personalization. Springer, 164--175.
Sophia B Liu. 2014. Crisis crowdsourcing framework: Designing strategic configurations of crowdsourcing for the emergency management domain. CSCW 23, 4--6 (2014), 389--443.
Nick Logler, Daisy Yoo, and Batya Friedman. 2018. Metaphor cards: A how-to-guide for making and using a generative metaphorical design toolkit. In Proceedings of the 2018 Designing Interactive Systems Conference. 1373--1386.
James Derek Lomas, Mihovil Karac, and Mathieu Gielen. 2021. Design Space Cards: Using a Card Deck to Navigate the Design Space of Interactive Play. Proceedings of the ACM on Human-Computer Interaction 5, CHIPLAY (2021), 227.
Andrés Lucero, Peter Dalsgaard, Kim Halskov, and Jacob Buur. 2016. Designing with cards. In Collaboration in creative design. Springer, 75--95.
Ewa Luger, Lachlan Urquhart, Tom Rodden, and Michael Golembewski. 2015. Playing the legal card: Using ideation cards to raise data protection issues within the design process. In Proceedings of the 33rd Annual ACM conference on human factors in computing systems. 457--466.
Mateus Monteiro, Leonardo Vasconcelos, José Viterbo, Luciana Salgado, and Flávia Bernardini. 2021. Assessing the Quality of Local E-Government Service Through Citizen-Sourcing Applications. In 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD). 1178--1183.
Simone Mora, Francesco Gianni, and Monica Divitini. 2017. Tiles: a card-based ideation toolkit for the internet of things. In Proceedings of the 2017 conference on designing interactive systems. 587--598.
Iohanna Nicenboim, Elisa Giaccardi, and Lenneke Kuijer. 2018. Designing connected resources for older people. In Proceedings of the 2018 Designing Interactive Systems Conference. 413--425.
Donald A Norman. 1999. Affordance, conventions, and design. interactions 6, 3 (1999), 38--43.
Heather L O'Brien and Elaine G Toms. 2010. The development and evaluation of a survey to measure user engagement. Journal of the American Society for Information Science and Technology 61, 1 (2010), 50--69.
Heather L O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112 (2018), 28--39.
Sangkeun Park, Sujin Kwon, and Uichin Lee. 2018. Campuswatch: Exploring community sourced patrolling with pervasive mobile technology. PACM HCI 2, CSCW (2018), 1--25.
David J Roedl and Erik Stolterman. 2013. Design research at CHI and its applicability to design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1951--1954.
Robin Roy and James P Warren. 2019. Card-based design tools: A review and analysis of 155 card decks for designers and designing. Design Studies 63 (2019), 125--154.
Lei Shi, James MacKrill, Elisavet Dimitrokali, and Rebecca Cain. 2015. Digital Crowdsourcing in Healthcare Environment Co-design. (2015).
Rabail Tahir and Alf Inge Wang. 2020. Transforming a theoretical framework to design cards: LEAGUE ideation toolkit for game-based learning design. Sustainability 12, 20 (2020), 8487.
Ramine Tinati, Max Van Kleek, Elena Simperl, Markus Luczak-Rösch, Robert Simpson, and Nigel Shadbolt. 2015. Designing for citizen data analysis: A cross-sectional case study of a multi-domain citizen science platform. In ACM Conference on Human Factors in Computing Systems. 4069--4078.
Leonardo Vasconcelos, Raissa Barcellos, José Viterbo, Flavia Bernardini, Clodis Boscarioli, and Eunice Nunes. 2019. How factors that influence engagement impact users' evaluations in mobile app stores. In International Conference on Human-Computer Interaction. Springer, 571--584.
Leonardo Vasconcelos, Daniela Trevisan, and José Viterbo. 2022. Engagement by Design: A Card-based approach to design crowdsourcing initiatives. In 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 353--358.
Publicado
17/10/2022
Como Citar
VASCONCELOS, Leonardo; TREVISAN, Daniela; VITERBO, José.
Applying the engagement by design methodology: perceptions and lessons learned. In: SIMPÓSIO BRASILEIRO SOBRE FATORES HUMANOS EM SISTEMAS COMPUTACIONAIS (IHC), 21. , 2022, Diamantina.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
