SPEAR: An End-to-End System for Designing, Conducting, and Analyzing Web-Based Studies
Resumo
Controlled experiments are widely used to study user behavior when interacting with Web systems, yet most existing tools are proprietary, require dedicated infrastructure, or target specific domains, limiting adoption by researchers with constrained resources. This paper presents SPEAR, an open-source tool for planning, executing and analyzing Web-based experiments, covering key stages of the experimental workflow, from participant recruitment to data visualization. The tool supports both between-subjects and within-subjects study designs, questionnaire-based group assignment, and self-hosted deployment without reliance on third-party services. To assess the system’s initial suitability for research use, we conducted a Think-Aloud study with 10 participants. As a proof of concept, the tool was instantiated for experiments involving chat tools and search interfaces, demonstrating the complete end-to-end workflow.
Referências
André Barcaui. 2025. ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention. Social Sciences & Humanities Open 12 (2025), 102287.
Ann Blandford, Anna L. Cox, and Paul Cairns. 2008. Controlled experiments. Cambridge University Press, 1–16.
John Brooke. 1996. Sus: a “quick and dirty’usability. Usability evaluation in industry 189, 3 (1996), 189–194.
K. Anders Ericsson and Herbert A. Simon. 1993. Protocol Analysis: Verbal Reports as Data (revised ed.). MIT Press, Cambridge, MA.
Barney G. Glaser and Anselm L. Strauss. 2017. The Discovery of Grounded Theory: Strategies for Qualitative Research. Routledge.
Felix Henninger, Yury Shevchenko, Ulf K. Mertens, Pascal J. Kieslich, and Benjamin E. Hilbig. 2022. lab.js: A free, open, online study builder. Behavior Research Methods 54, 2 (2022), 556–573. DOI: 10.3758/s13428-021-01548-7
HN de O Joel, Alfredo Neto, Breno Rosado, Marcelo Machado, Jairo F de Souza, and Sean WM Siqueira. 2025. A Framework for Search as Learning Experiments: Design, Implementation, and Usability Insights. In Simpósio Brasileiro de Informática na Educação (SBIE). SBC, 304–315.
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Vol. 2. Morgan Kaufmann Publishers Cambridge, MA.
Thiago J. Lopes, Yeshua Siqueira, Jairo F. de Souza, and Marcelo Machado. 2026. Traditional vs. Socratic: How Chatbot Interaction Strategies Shape Self-Directed Learning?. In Anais do XXXVII Simpósio Brasileiro de Informática na Educação (SBIE). SBC.
Ulf-Dietrich Reips and Christoph Neuhaus. 2002. WEXTOR: A Web-based tool for generating and visualizing experimental designs and procedures. Behavior Research Methods, Instruments, & Computers 34, 2 (2002), 234–240.
William R Shadish. 2002. Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage Learning (2002).
Yury Shevchenko. 2022. Open Lab: A web application for running and sharing online experiments. Behavior Research Methods 54 (03 2022), 3118–3125. DOI: 10.3758/s13428-021-01776-2
