Evaluating the Development Experience in Open Science Tools: A Case Study of the dataWASHES API
Abstract
Open science has gained prominence in Software Engineering research, yet Developer Experience (DX) in open science tools remains an underexplored topic. Recognizing this relevance, this study investigates developers’ perceptions of DX when using dataWASHES, an open source API that facilitates access to WASHES proceedings data. A qualitative study was conducted with 13 participants performing scenario-based tasks using the API, followed by a questionnaire inspired by the SPACE framework. Results indicate a positive DX, with 84.6% of participants rating their experience at the highest level. While satisfaction and efficiency were strengths, minor usability issues suggest areas for improvement. Analyzing DX in tools like dataWASHES advances our understanding of the relationship between open science tools and DX, providing important lessons towards the engineering of open science solutions.
References
Araújo, A. A., Farias, I., Gonçalves, V., Santos, R., Viana, D., and Steinmacher, I. (2024) dataWASHES - Towards an Application Programming Interface of WASHES proceedings data. In Workshop sobre Aspectos Sociais, Humanos e Econômicos de Software (WASHES), pages 158–162. SBC.
Bevan, N., Barnum, C., Cockton, G., Nielsen, J., Spool, J., and Wixon, D. (2003). The “magic number 5” is it enough for web testing? In CHI’03 extended abstracts on Human factors in computing systems, pages 698–699.
Fagerholm, F. and Münch, J. (2012). Developer experience: Concept and definition. In 2012 international conference on software and system process (ICSSP), pages 73–77. IEEE.
Fernández, D. M., Monperrus, M., Feldt, R., and Zimmermann, T. (2019). The open science initiative of the empirical software engineering journal. Empirical Software Engineering, 24:1057–1060.
Forsgren, N., Kalliamvakou, E., Noda, A., Greiler, M., Houck, B., and Storey, M.-A. (2024). Devex in action. Communications of the ACM, 67(6):42–51.
Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., and Butler, J. (2021). The space of developer productivity: There’s more to it than you think. Queue, 19(1):20–48.
Greiler, M., Storey, M.-A., and Noda, A. (2022). An actionable framework for understanding and improving developer experience. IEEE Transactions on Software Engineering, 49(4):1411–1425.
Guenes, P., Tomaz, R., Kalinowski, M., Baldassarre, M. T., and Storey, M.-A. (2024). Impostor phenomenon in software engineers. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Society, pages 96–106.
Guest, G., Bunce, A., and Johnson, L. (2006). How many interviews are enough? an experiment with data saturation and variability. Field methods, 18(1):59–82.
Hicks, C. M. (2024). Psychological affordances can provide a missing explanatory layer for why interventions to improve developer experience take hold or fail. Preprint. DOI: 10.31234/osf.io/qz43x.
Kags, A. (2023). Building community driven open infrastructure. In Open Infrastructure Fund/Fondo de Infraestructura Abierta.
Mendez, D., Graziotin, D., Wagner, S., and Seibold, H. (2020). Open science in software engineering. Contemporary empirical methods in software engineering, pages 477–501.
Mikkonen, T. (2016). Flow, intrinsic motivation, and developer experience in software engineering. Agile processes in software engineering and extreme programming, 104.
Nylund, A. (2020). A multivocal literature review on developer experience. Master’s thesis, Aalto University.
Persic, A., Beigel, F., Hodson, S., and Oti-Boateng, P. (2021). The time for open science is now. UNESCO Science Report: the race against time for smarter development. Paris: UNESCO Publishing, pages 12–16.
Peršić, A. and Straza, T. (2023). Open science for all: Implementing the unesco recommendation on open science for an equitable and just transition to open science. College & Research Libraries News, 84(10):377.
Razzaq, A., Buckley, J., Lai, Q., Yu, T., and Botterweck, G. (2024). A systematic literature review on the influence of enhanced developer experience on developers’ productivity: Factors, practices, and recommendations. ACM Computing Surveys, 57(1):1–46.
Runeson, P. and Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering, 14:131–164.
Sellanga, J. (2023). Accelerating adoption and investment in open infrastructure world-wide through collaboration with networks. Invest in Open. Accessed: 2024-03-14.
Sikand, S., Phokela, K. K., Sharma, V. S., Singi, K., Kaulgud, V., Tung, T., Sharma, P., and Burden, A. P. (2024). How much space do metrics have in genai assisted software development? In Proceedings of the 17th Innovations in Software Engineering Conference, pages 1–5.
Souza, A., Viana, D., Santos, R., and Araújo, A. A. (2024). Supporting dataset. DOI: 10.5281/zenodo.15061563.
UNESCO (2021). UNESCO Recommendation on Open Science. United Nations Educational, Scientific and Cultural Organization.
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., and Aftandilian, E. (2024). Measuring github copilot’s impact on productivity. Communications of the ACM, 67(3):54–63.
Bevan, N., Barnum, C., Cockton, G., Nielsen, J., Spool, J., and Wixon, D. (2003). The “magic number 5” is it enough for web testing? In CHI’03 extended abstracts on Human factors in computing systems, pages 698–699.
Fagerholm, F. and Münch, J. (2012). Developer experience: Concept and definition. In 2012 international conference on software and system process (ICSSP), pages 73–77. IEEE.
Fernández, D. M., Monperrus, M., Feldt, R., and Zimmermann, T. (2019). The open science initiative of the empirical software engineering journal. Empirical Software Engineering, 24:1057–1060.
Forsgren, N., Kalliamvakou, E., Noda, A., Greiler, M., Houck, B., and Storey, M.-A. (2024). Devex in action. Communications of the ACM, 67(6):42–51.
Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., and Butler, J. (2021). The space of developer productivity: There’s more to it than you think. Queue, 19(1):20–48.
Greiler, M., Storey, M.-A., and Noda, A. (2022). An actionable framework for understanding and improving developer experience. IEEE Transactions on Software Engineering, 49(4):1411–1425.
Guenes, P., Tomaz, R., Kalinowski, M., Baldassarre, M. T., and Storey, M.-A. (2024). Impostor phenomenon in software engineers. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Society, pages 96–106.
Guest, G., Bunce, A., and Johnson, L. (2006). How many interviews are enough? an experiment with data saturation and variability. Field methods, 18(1):59–82.
Hicks, C. M. (2024). Psychological affordances can provide a missing explanatory layer for why interventions to improve developer experience take hold or fail. Preprint. DOI: 10.31234/osf.io/qz43x.
Kags, A. (2023). Building community driven open infrastructure. In Open Infrastructure Fund/Fondo de Infraestructura Abierta.
Mendez, D., Graziotin, D., Wagner, S., and Seibold, H. (2020). Open science in software engineering. Contemporary empirical methods in software engineering, pages 477–501.
Mikkonen, T. (2016). Flow, intrinsic motivation, and developer experience in software engineering. Agile processes in software engineering and extreme programming, 104.
Nylund, A. (2020). A multivocal literature review on developer experience. Master’s thesis, Aalto University.
Persic, A., Beigel, F., Hodson, S., and Oti-Boateng, P. (2021). The time for open science is now. UNESCO Science Report: the race against time for smarter development. Paris: UNESCO Publishing, pages 12–16.
Peršić, A. and Straza, T. (2023). Open science for all: Implementing the unesco recommendation on open science for an equitable and just transition to open science. College & Research Libraries News, 84(10):377.
Razzaq, A., Buckley, J., Lai, Q., Yu, T., and Botterweck, G. (2024). A systematic literature review on the influence of enhanced developer experience on developers’ productivity: Factors, practices, and recommendations. ACM Computing Surveys, 57(1):1–46.
Runeson, P. and Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering, 14:131–164.
Sellanga, J. (2023). Accelerating adoption and investment in open infrastructure world-wide through collaboration with networks. Invest in Open. Accessed: 2024-03-14.
Sikand, S., Phokela, K. K., Sharma, V. S., Singi, K., Kaulgud, V., Tung, T., Sharma, P., and Burden, A. P. (2024). How much space do metrics have in genai assisted software development? In Proceedings of the 17th Innovations in Software Engineering Conference, pages 1–5.
Souza, A., Viana, D., Santos, R., and Araújo, A. A. (2024). Supporting dataset. DOI: 10.5281/zenodo.15061563.
UNESCO (2021). UNESCO Recommendation on Open Science. United Nations Educational, Scientific and Cultural Organization.
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., and Aftandilian, E. (2024). Measuring github copilot’s impact on productivity. Communications of the ACM, 67(3):54–63.
Published
2025-07-20
How to Cite
SOUZA, Antônio; VIANA, Davi; SANTOS, Rodrigo; ARAÚJO, Allysson Allex.
Evaluating the Development Experience in Open Science Tools: A Case Study of the dataWASHES API. In: PROCEEDINGS OF WORKSHOP ON SOCIAL, HUMAN AND ECONOMIC ASPECTS OF SOFTWARE (WASHES), 10. , 2025, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 36-47.
ISSN 2763-874X.
DOI: https://doi.org/10.5753/washes.2025.8006.
