A Literature Study to Characterize Continuous Experimentation in Software Engineering
Resumo
Continuous Experimentation (CE) has become increasingly popular across industry and academic communities. Given its rapid evolution in software engineering (SE), the lack of a common understanding of CE can jeopardize new implementations and justify research efforts. Therefore, this literature study characterizes CE in SE based on its definitions, processes, and strategies for experimentation available in the technical literature. Seventy-six sources of information provided many different definitions, processes, and experimental procedures used to describe CE in SE. Despite the increasing use of CE in SE, it is impossible to observe a common terminology yet to support its characterization and use.
Palavras-chave:
Continuous Experimentation, Continuous Software Engineering, Literature Review
Referências
Basili, V. R., Caldiera, G., Rombach, H. D. (1994) "The goal question metric approach"; Encyclopedia of software engineering, 528-532.
Kuhrmann, M., Fernández, D. M., Daneva, M. (2017) “On the pragmatic design of literature studies in software engineering: an experience-based guideline”; ESE 22.6.
Wohlin, C. (2014) "Guidelines for snowballing in systematic literature studies and a replication in software engineering"; Proceedings of the 18th EASE.
Auer, F. and Felderer, M. (2018) "Current state of research on continuous experimentation: a systematic mapping study"; 2018 44th Euromicro SEAA, IEEE.
Auer, F., Lee C. S., and Felderer, M. (2020) "Continuous experiment definition characteristics"; 2020 46th Euromicro SEAA, IEEE.
Auer, F. et al. (2021) "Controlled experimentation in continuous experimentation: Knowledge and challenges"; Information and Software Technology 134, 106551.
Bosch, J., Eklund, U. (2012) "Eternal embedded software: Towards innovation experiment systems"; ISoLA, Springer, Berlin, Heidelberg.
Bosch, J. and Olsson, H. H. (2016) "Data-driven continuous evolution of smart systems"; 2016 IEEE/ACM 11th SEAMS, IEEE.
Bosch, J., and Olsson, H. H. (2017) "Toward evidence-based organizations: lessons from embedded systems, online games, and the Internet of Things"; IEEE Software 34.5.
Bosch, J. (2012) "Building products as innovation experiment systems"; International Conference of Software Business, Springer, Berlin, Heidelberg.
Bosch?Sijtsema, P. and Bosch, J. (2015) "User involvement throughout the innovation process in high?tech industries"; JPIM 32.5, p. 793-807.
Buchert, T. et al. (2015) "A survey of general-purpose experiment management tools for distributed systems"; Future Generation Computer Systems 45, p. 1-12.
Crook, T., et al. (2009) "Seven pitfalls to avoid when running controlled experiments on the web"; 15th ACM SIGKDD.
Deng, A., and Shi, X. (2016) "Data-driven metric development for online controlled experiments: Seven lessons"; 22nd ACM SIGKDD.
Eklund, U. and Bosch, J. (2012) "Architecture for large-scale innovation experiment systems"; IEEE/IFIP Conference and European Conference, IEEE.
Esteller-Cucala, M., Fernandez, V., and Villuendas, D. (2020) "Towards data-driven culture in a Spanish automobile manufacturer: A case study"; JIEM 13.2.
Fabijan, A., et al. (2017) "The evolution of continuous experimentation in software product development: from data to a data-driven organization at scale"; IEEE/ACM 39th ICSE, IEEE.
Fabijan, A., et al. (2018a) "Experimentation growth: Evolving trustworthy A/B testing capabilities in online software companies"; J. Softw.: Evol. Process 30.12.
Fabijan, A. et al. (2018b) "Online controlled experimentation at scale: an empirical survey on the current state of A/B testing"; 44th Euromicro SEAA, IEEE.
Fabijan, A. et al. (2018c) "The online controlled experiment lifecycle"; IEEE Software 37.2, p. 60-67.
Fabijan, A., et al. (2019) "Three key checklists and remedies for trustworthy analysis of online controlled experiments at scale"; IEEE/ACM ICSE-SEIP, IEEE.
Fagerholm, F. et al. (2014) "Building blocks for continuous experimentation"; 1st international workshop on rapid continuous software engineering.
Fagerholm, F., et al. (2017) "The RIGHT model for continuous experimentation"; Journal of Systems and Software 123, p. 292-305.
Feitelson, D. G., Frachtenberg, E. and Beck, K. L. (2013) "Development and deployment at Facebook"; IEEE Internet Computing 17.4, p. 8-17.
Fitzgerald, B. and Stol, K.J. (2017) "Continuous software engineering: A roadmap and agenda"; Journal of Systems and Software 123, p. 176-189.
Gerostathopoulos, I., et al. (2018) "Cost-aware stage-based experimentation: challenges and emerging results"; 2018 IEEE ICSA-C, IEEE.
Giaimo, F. and Berger, C. (2017) "Design criteria to architect continuous experimentation for self-driving vehicles"; IEEE ICSA.
Giaimo, F. and Berger, C. (2020) "Continuous Experimentation for Automotive Software on the Example of a Heavy Commercial Vehicle in Daily Operation"; ECSA.
Giaimo, F., et al. (2016) "Continuous experimentation on cyber-physical systems: challenges and opportunities"; Proceedings of the scientific workshop XP2016.
Giaimo, F., Berger, C., and Kirchner, C. (2017) "Considerations about continuous experimentation for resource-constrained platforms in self-driving vehicles"; ECSA.
Giaimo, F., Andrade, H. and Berger, C. (2019) "The automotive take on continuous experimentation: a multiple case study"; 45th SEAA, IEEE.
Giaimo, F., Andrade, H. and Berger, C. (2020) "Continuous experimentation and the cyber-physical systems challenge: An overview of the literature and the industrial perspective"; Journal of Systems and Software 170, 110781.
Gomez-Uribe, C. A. and Hunt, N. (2015) "The Netflix recommender system: Algorithms, business value, and innovation"; ACM TMIS 6.4, p. 1-19.
Gupta, S. et al. (2019) "Top challenges from the first practical online controlled experiments summit"; ACM SIGKDD Explorations Newsletter 21.1, p. 20-35.
Gutbrod, M., Münch, J. and Tichy, M. (2019) "How do software startups approach experimentation? Empirical results from a qualitative interview study"; PROFES.
Jiang, S., Martin, J. and Wilson, C. (2019) "Who's the Guinea Pig? Investigating Online A/B/n Tests in-the-Wild"; Proceedings of the ACM FAccT.
Karvonen, T. et al. (2015) "Hitting the Target: Practices and Steps for Moving Towards Innovation Experiment Systems"; LNBIP.
Kevic, K. et al. (2017) "Characterizing experimentation in continuous deployment: a case study on bing"; 2017 IEEE/ACM 39th ICSE-SEIP, IEEE.
Kohavi, R., Henne, R. M. and Sommerfield, D. (2007) "Practical guide to controlled experiments on the web: listen to your customers, not to the hippo"; 13th ACM SIGKDD.
Kohavi, R., et al. (2012) "Trustworthy online controlled experiments: Five puzzling outcomes explained"; 18th ACM SIGKDD.
Kohavi, R., et al. (2013) "Online controlled experiments at large scale"; 19th ACM SIGKDD.
Kohavi, R., et al. (2014) "Seven rules of thumb for website experimenters"; 20th ACM SIGKDD.
Lindgren, E. and Münch, J. (2015) "Software development as an experiment system: A qualitative survey on the state of the practice"; ICASD.
Lindgren, E. and Münch, J. (2016) "Raising the odds of success: the current state of experimentation in product development"; IST 77, p. 80-91.
Liu, S. et al. (2019) "Enterprise-Level Controlled Experiments at Scale: Challenges and Solutions"; 45th Euromicro Conference on SEAA, IEEE.
Mattos, D. I., et al. (2018a) "An activity and metric model for online controlled experiments"; PROFES, Springer, Cham.
Mattos, D. I., Bosch, J., and Olsson, H. H. (2018b) "Challenges and strategies for undertaking continuous experimentation to embedded systems: Industry and research perspectives"; ICASD.
Mattos, D. I., et al. (2020a) "Automotive a/b testing: Challenges and lessons learned from practice"; 46th Euromicro Conference on SEAA, IEEE.
Mattos, D. I., et al. (2020b) "Experimentation for business-to-business mission-critical systems: a case study"; Proceedings of the ICSSP.
Melegati, J. and Wang, X. (2020) "Hypotheses Elicitation in Early-Stage Software Startups Based on Cognitive Mapping"; ICASD, Springer, Cham.
Melegati, J., Wang, X., and Abrahamsson, P. (2019a) "Hypotheses Engineering: first essential steps of experiment-driven software development"; 4th International Workshop on RCoSE/DDrEE, IEEE.
Melegati, J., et al. (2019b) "Enablers and inhibitors of experimentation in early-stage software startups"; PROFES, Springer, Cham.
Melegati, J., Edison, H., and Wang, X. (2020a) "XPro: a Model to Explain the Limited Adoption and Implementation of Experimentation in Software Startups"; IEEE TOSEM.
Melegati, J., et al. (2020b) "MVP and experimentation in software startups: a qualitative survey"; 46th Euromicro Conference on SEAA, IEEE.
Melegati, J., Guerra, E., and Wang, X. (2020c) "Understanding Hypotheses Engineering in Software Startups through a Gray Literature Review"; IST, 106465.
Melegati, J. (2019) "Improving requirements engineering practices to support experimentation in software startups"; 27th ACM Joint Meeting on ESEC/FSE.
Olsson, H. H. and Bosch, J. (2013a) "Post-deployment data collection in software-intensive embedded products"; ICSOB, Springer, Berlin, Heidelberg.
Olsson, H. H. and Bosch, J. (2013b) "Towards data-driven product development: A multiple case study on post-deployment data usage in software-intensive embedded systems"; LESS, Springer, Berlin, Heidelberg.
Olsson, H. H. and Bosch, J. (2014) "From opinions to data-driven software R&D: A multi-case study on how to close the 'open loop' problem"; 40th EUROMICRO SEAA.
Olsson, H. H. and Bosch, J. (2015a) "Towards continuous validation of customer value"; Scientific Workshop Proceedings of the XP2015.
Olsson, H. H. and Bosch, J. (2015b) "Towards continuous customer validation: A conceptual model for combining qualitative customer feedback with quantitative customer observation"; International Conference of Software Business. Springer, Cham.
Olsson, H. H. and Bosch, J. (2019) "Data-driven development: Challenges in online, embedded and on-premise software"; PROFES, Springer, Cham.
Olsson, H. H., Bosch, J. and Alahyari, H. (2013) "Towards R&D as innovation experiment systems: A framework for moving beyond agile software development"; IASTED ().
Olsson, H. H., Bosch, J. and Fabijan, A. (2017) "Experimentation that matters: a multi-case study on the challenges with A/B testing"; ICSOB, Springer, Cham.
Rissanen, O. and Münch, J. (2015) "Continuous experimentation in the B2B domain: a case study"; IEEE/ACM 2nd International Workshop on RCoSE, IEEE.
Ros, R. and Bjarnason, E. (2018) "Continuous experimentation scenarios: a case study in e-Commerce"; 44th Euromicro Conference on SEAA, IEEE.
Ros, R. and Runeson, P. (2018) "Continuous experimentation and a/b testing: A mapping study"; 2018 IEEE/ACM 4th International Workshop on (RCoSE), IEEE.
Ros, R. (2020) "Continuous Experimentation with Product-Led Business Models: A Comparative Case Study"; ICSOB, Springer, Cham.
Sauvola, T. et al. (2015) "Towards customer-centric software development: a multiple-case study"; 41st Euromicro Conference on SEAA, IEEE.
Schermann, G. et al. (2016) "Bifrost: Supporting continuous deployment with automated enactment of multi-phase live testing strategies"; 17th IMC.
Schermann, G., Cito, J., and Leitner, P. (2018a) "Continuous experimentation: challenges, implementation techniques, and current research"; Ieee Software 35.2, p. 26-31.
Schermann, G., et al. (2018b) "We’re doing it live: A multi-method empirical study on continuous experimentation”; Information and Software Technology, v. 99.
Sveningson, R., Mattos, D. I. and Bosch, J. (2019) “Continuous experimentation for software organizations with low control of roadmap and a large distance to users: An exploratory case study”; PROFES, Springer, Cham.
Tang, D., et al. (2010) “Overlapping experiment infrastructure: More, better, faster experimentation”; 16th ACM SIGKDD International Conference on KDD.
Xu, Y. and Chen, N. (2016) “Evaluating mobile apps with A/B and quasi A/B tests”; 22nd ACM SIGKDD International Conference on KDD.
Yaman, S. G., et al. (2016) “Transitioning towards continuous experimentation in a large software product and service development organization–a case study”; PROFES.
Yaman, S. et al. (2017) “Notifying and involving users in experimentation: ethical perceptions of software practitioners”; ESEM.
Yaman, S., Mikkonen, T., and Suomela, R. (2018) “Continuous experimentation in mobile game development”; 2018 44th Euromicro Conference on SEAA, IEEE.
Yaman, S., et al. (2020) “Patterns of user involvement in experiment-driven software development”; Information and Software Technology, v. 120, p. 106244.
Kuhrmann, M., Fernández, D. M., Daneva, M. (2017) “On the pragmatic design of literature studies in software engineering: an experience-based guideline”; ESE 22.6.
Wohlin, C. (2014) "Guidelines for snowballing in systematic literature studies and a replication in software engineering"; Proceedings of the 18th EASE.
Auer, F. and Felderer, M. (2018) "Current state of research on continuous experimentation: a systematic mapping study"; 2018 44th Euromicro SEAA, IEEE.
Auer, F., Lee C. S., and Felderer, M. (2020) "Continuous experiment definition characteristics"; 2020 46th Euromicro SEAA, IEEE.
Auer, F. et al. (2021) "Controlled experimentation in continuous experimentation: Knowledge and challenges"; Information and Software Technology 134, 106551.
Bosch, J., Eklund, U. (2012) "Eternal embedded software: Towards innovation experiment systems"; ISoLA, Springer, Berlin, Heidelberg.
Bosch, J. and Olsson, H. H. (2016) "Data-driven continuous evolution of smart systems"; 2016 IEEE/ACM 11th SEAMS, IEEE.
Bosch, J., and Olsson, H. H. (2017) "Toward evidence-based organizations: lessons from embedded systems, online games, and the Internet of Things"; IEEE Software 34.5.
Bosch, J. (2012) "Building products as innovation experiment systems"; International Conference of Software Business, Springer, Berlin, Heidelberg.
Bosch?Sijtsema, P. and Bosch, J. (2015) "User involvement throughout the innovation process in high?tech industries"; JPIM 32.5, p. 793-807.
Buchert, T. et al. (2015) "A survey of general-purpose experiment management tools for distributed systems"; Future Generation Computer Systems 45, p. 1-12.
Crook, T., et al. (2009) "Seven pitfalls to avoid when running controlled experiments on the web"; 15th ACM SIGKDD.
Deng, A., and Shi, X. (2016) "Data-driven metric development for online controlled experiments: Seven lessons"; 22nd ACM SIGKDD.
Eklund, U. and Bosch, J. (2012) "Architecture for large-scale innovation experiment systems"; IEEE/IFIP Conference and European Conference, IEEE.
Esteller-Cucala, M., Fernandez, V., and Villuendas, D. (2020) "Towards data-driven culture in a Spanish automobile manufacturer: A case study"; JIEM 13.2.
Fabijan, A., et al. (2017) "The evolution of continuous experimentation in software product development: from data to a data-driven organization at scale"; IEEE/ACM 39th ICSE, IEEE.
Fabijan, A., et al. (2018a) "Experimentation growth: Evolving trustworthy A/B testing capabilities in online software companies"; J. Softw.: Evol. Process 30.12.
Fabijan, A. et al. (2018b) "Online controlled experimentation at scale: an empirical survey on the current state of A/B testing"; 44th Euromicro SEAA, IEEE.
Fabijan, A. et al. (2018c) "The online controlled experiment lifecycle"; IEEE Software 37.2, p. 60-67.
Fabijan, A., et al. (2019) "Three key checklists and remedies for trustworthy analysis of online controlled experiments at scale"; IEEE/ACM ICSE-SEIP, IEEE.
Fagerholm, F. et al. (2014) "Building blocks for continuous experimentation"; 1st international workshop on rapid continuous software engineering.
Fagerholm, F., et al. (2017) "The RIGHT model for continuous experimentation"; Journal of Systems and Software 123, p. 292-305.
Feitelson, D. G., Frachtenberg, E. and Beck, K. L. (2013) "Development and deployment at Facebook"; IEEE Internet Computing 17.4, p. 8-17.
Fitzgerald, B. and Stol, K.J. (2017) "Continuous software engineering: A roadmap and agenda"; Journal of Systems and Software 123, p. 176-189.
Gerostathopoulos, I., et al. (2018) "Cost-aware stage-based experimentation: challenges and emerging results"; 2018 IEEE ICSA-C, IEEE.
Giaimo, F. and Berger, C. (2017) "Design criteria to architect continuous experimentation for self-driving vehicles"; IEEE ICSA.
Giaimo, F. and Berger, C. (2020) "Continuous Experimentation for Automotive Software on the Example of a Heavy Commercial Vehicle in Daily Operation"; ECSA.
Giaimo, F., et al. (2016) "Continuous experimentation on cyber-physical systems: challenges and opportunities"; Proceedings of the scientific workshop XP2016.
Giaimo, F., Berger, C., and Kirchner, C. (2017) "Considerations about continuous experimentation for resource-constrained platforms in self-driving vehicles"; ECSA.
Giaimo, F., Andrade, H. and Berger, C. (2019) "The automotive take on continuous experimentation: a multiple case study"; 45th SEAA, IEEE.
Giaimo, F., Andrade, H. and Berger, C. (2020) "Continuous experimentation and the cyber-physical systems challenge: An overview of the literature and the industrial perspective"; Journal of Systems and Software 170, 110781.
Gomez-Uribe, C. A. and Hunt, N. (2015) "The Netflix recommender system: Algorithms, business value, and innovation"; ACM TMIS 6.4, p. 1-19.
Gupta, S. et al. (2019) "Top challenges from the first practical online controlled experiments summit"; ACM SIGKDD Explorations Newsletter 21.1, p. 20-35.
Gutbrod, M., Münch, J. and Tichy, M. (2019) "How do software startups approach experimentation? Empirical results from a qualitative interview study"; PROFES.
Jiang, S., Martin, J. and Wilson, C. (2019) "Who's the Guinea Pig? Investigating Online A/B/n Tests in-the-Wild"; Proceedings of the ACM FAccT.
Karvonen, T. et al. (2015) "Hitting the Target: Practices and Steps for Moving Towards Innovation Experiment Systems"; LNBIP.
Kevic, K. et al. (2017) "Characterizing experimentation in continuous deployment: a case study on bing"; 2017 IEEE/ACM 39th ICSE-SEIP, IEEE.
Kohavi, R., Henne, R. M. and Sommerfield, D. (2007) "Practical guide to controlled experiments on the web: listen to your customers, not to the hippo"; 13th ACM SIGKDD.
Kohavi, R., et al. (2012) "Trustworthy online controlled experiments: Five puzzling outcomes explained"; 18th ACM SIGKDD.
Kohavi, R., et al. (2013) "Online controlled experiments at large scale"; 19th ACM SIGKDD.
Kohavi, R., et al. (2014) "Seven rules of thumb for website experimenters"; 20th ACM SIGKDD.
Lindgren, E. and Münch, J. (2015) "Software development as an experiment system: A qualitative survey on the state of the practice"; ICASD.
Lindgren, E. and Münch, J. (2016) "Raising the odds of success: the current state of experimentation in product development"; IST 77, p. 80-91.
Liu, S. et al. (2019) "Enterprise-Level Controlled Experiments at Scale: Challenges and Solutions"; 45th Euromicro Conference on SEAA, IEEE.
Mattos, D. I., et al. (2018a) "An activity and metric model for online controlled experiments"; PROFES, Springer, Cham.
Mattos, D. I., Bosch, J., and Olsson, H. H. (2018b) "Challenges and strategies for undertaking continuous experimentation to embedded systems: Industry and research perspectives"; ICASD.
Mattos, D. I., et al. (2020a) "Automotive a/b testing: Challenges and lessons learned from practice"; 46th Euromicro Conference on SEAA, IEEE.
Mattos, D. I., et al. (2020b) "Experimentation for business-to-business mission-critical systems: a case study"; Proceedings of the ICSSP.
Melegati, J. and Wang, X. (2020) "Hypotheses Elicitation in Early-Stage Software Startups Based on Cognitive Mapping"; ICASD, Springer, Cham.
Melegati, J., Wang, X., and Abrahamsson, P. (2019a) "Hypotheses Engineering: first essential steps of experiment-driven software development"; 4th International Workshop on RCoSE/DDrEE, IEEE.
Melegati, J., et al. (2019b) "Enablers and inhibitors of experimentation in early-stage software startups"; PROFES, Springer, Cham.
Melegati, J., Edison, H., and Wang, X. (2020a) "XPro: a Model to Explain the Limited Adoption and Implementation of Experimentation in Software Startups"; IEEE TOSEM.
Melegati, J., et al. (2020b) "MVP and experimentation in software startups: a qualitative survey"; 46th Euromicro Conference on SEAA, IEEE.
Melegati, J., Guerra, E., and Wang, X. (2020c) "Understanding Hypotheses Engineering in Software Startups through a Gray Literature Review"; IST, 106465.
Melegati, J. (2019) "Improving requirements engineering practices to support experimentation in software startups"; 27th ACM Joint Meeting on ESEC/FSE.
Olsson, H. H. and Bosch, J. (2013a) "Post-deployment data collection in software-intensive embedded products"; ICSOB, Springer, Berlin, Heidelberg.
Olsson, H. H. and Bosch, J. (2013b) "Towards data-driven product development: A multiple case study on post-deployment data usage in software-intensive embedded systems"; LESS, Springer, Berlin, Heidelberg.
Olsson, H. H. and Bosch, J. (2014) "From opinions to data-driven software R&D: A multi-case study on how to close the 'open loop' problem"; 40th EUROMICRO SEAA.
Olsson, H. H. and Bosch, J. (2015a) "Towards continuous validation of customer value"; Scientific Workshop Proceedings of the XP2015.
Olsson, H. H. and Bosch, J. (2015b) "Towards continuous customer validation: A conceptual model for combining qualitative customer feedback with quantitative customer observation"; International Conference of Software Business. Springer, Cham.
Olsson, H. H. and Bosch, J. (2019) "Data-driven development: Challenges in online, embedded and on-premise software"; PROFES, Springer, Cham.
Olsson, H. H., Bosch, J. and Alahyari, H. (2013) "Towards R&D as innovation experiment systems: A framework for moving beyond agile software development"; IASTED ().
Olsson, H. H., Bosch, J. and Fabijan, A. (2017) "Experimentation that matters: a multi-case study on the challenges with A/B testing"; ICSOB, Springer, Cham.
Rissanen, O. and Münch, J. (2015) "Continuous experimentation in the B2B domain: a case study"; IEEE/ACM 2nd International Workshop on RCoSE, IEEE.
Ros, R. and Bjarnason, E. (2018) "Continuous experimentation scenarios: a case study in e-Commerce"; 44th Euromicro Conference on SEAA, IEEE.
Ros, R. and Runeson, P. (2018) "Continuous experimentation and a/b testing: A mapping study"; 2018 IEEE/ACM 4th International Workshop on (RCoSE), IEEE.
Ros, R. (2020) "Continuous Experimentation with Product-Led Business Models: A Comparative Case Study"; ICSOB, Springer, Cham.
Sauvola, T. et al. (2015) "Towards customer-centric software development: a multiple-case study"; 41st Euromicro Conference on SEAA, IEEE.
Schermann, G. et al. (2016) "Bifrost: Supporting continuous deployment with automated enactment of multi-phase live testing strategies"; 17th IMC.
Schermann, G., Cito, J., and Leitner, P. (2018a) "Continuous experimentation: challenges, implementation techniques, and current research"; Ieee Software 35.2, p. 26-31.
Schermann, G., et al. (2018b) "We’re doing it live: A multi-method empirical study on continuous experimentation”; Information and Software Technology, v. 99.
Sveningson, R., Mattos, D. I. and Bosch, J. (2019) “Continuous experimentation for software organizations with low control of roadmap and a large distance to users: An exploratory case study”; PROFES, Springer, Cham.
Tang, D., et al. (2010) “Overlapping experiment infrastructure: More, better, faster experimentation”; 16th ACM SIGKDD International Conference on KDD.
Xu, Y. and Chen, N. (2016) “Evaluating mobile apps with A/B and quasi A/B tests”; 22nd ACM SIGKDD International Conference on KDD.
Yaman, S. G., et al. (2016) “Transitioning towards continuous experimentation in a large software product and service development organization–a case study”; PROFES.
Yaman, S. et al. (2017) “Notifying and involving users in experimentation: ethical perceptions of software practitioners”; ESEM.
Yaman, S., Mikkonen, T., and Suomela, R. (2018) “Continuous experimentation in mobile game development”; 2018 44th Euromicro Conference on SEAA, IEEE.
Yaman, S., et al. (2020) “Patterns of user involvement in experiment-driven software development”; Information and Software Technology, v. 120, p. 106244.
Publicado
13/06/2022
Como Citar
ERTHAL, Vladimir M.; DE SOUZA, Bruno P. de; SANTOS, Paulo Sérgio M. dos; TRAVASSOS, Guilherme H..
A Literature Study to Characterize Continuous Experimentation in Software Engineering. In: CONGRESSO IBERO-AMERICANO EM ENGENHARIA DE SOFTWARE (CIBSE), 25. , 2022, Córdoba.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 1-15.
DOI: https://doi.org/10.5753/cibse.2022.20959.
