Factores Personales que Influyen en la Realización de Tareas bajo Test-Driven Development: Meta-Análisis de una Familia de Experimentos
Resumo
Antecedentes: La experimentación en Ingeniería del Software requiere frecuentemente que los sujetos experimentales realicen algún tipo de tarea. Objetivo: Estudiar el grado en que se realizan las tareas experimentales, en función de los aspectos instrumentales de un experimento y de las características personales de los sujetos. Método: Meta-análisis de una familia de experimentos en Test-Driven Development. Resultados: Los sujetos de mayor edad y más experiencia profesional realizan menos trabajo en las tareas experimentales. La tarea experimental puede influir en el grado de participación en el experimento. Conclusiones: Deben estudiarse las motivaciones de los sujetos experimentales y establecer mecanismos de control que aseguren un correcto desempeño.
Palavras-chave:
Experimentación, Sujetos experimentales, Tarea experimental, Conformidad, Completitud, Incentivos
Referências
Beck, K. (2003). Test Driven Development: By Example. Addison-Wesley.
Bissi, W., Neto, A. G. S. S., and Emer, M. C. F. P. (2016a). The effects of test driven development on internal quality, external quality and productivity: A systematic review. Information and Software Technology, 74:45–54.
Bissi, W., Scanniello, G., Romano, S., and Tortora, G. (2016b). On the perception of test driven development: A replicated study. Journal of Visual Languages & Computing, 34:11–24.
Chandler, J., Mueller, P., and Paolacci, G. (2014). Nonnaïveté among amazon mechanical turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1):112–130.
Deci, E. L. and Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Plenum, New York.
Diefendorff, J. M. and Richard, E. M. (2000). Antecedents and consequences of emotional display rule perceptions. Journal of Applied Psychology, 88(2):284–294.
Eysenck, M. W., Derakshan, N., Santos, R., and Calvo, M. G. (2007). Anxiety and cognitive performance: Attentional control theory. Emotion, 7(2):336–353.
Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., and Juristo, N. (2016). A dissection of the test-driven development process: does it really matter to test-first or to test-last? IEEE Transactions on Software Engineering, 43(7):597–614.
Fucci, D., Turhan, B., and Oivo, M. (2014). Impact of process conformance on the effects of test-driven development. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pages 1–10.
George, B. and Williams, L. (2004). A structured experiment of test-driven development. Information and Software Technology, 46(5):337–342.
Hauser, D. J. and Schwarz, N. (2018). Attentive turkers: Mturk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1):400–407.
Juristo, N. (2016). Experiences conducting experiments in industry: The eseil fidipro project. In 2016 IEEE/ACM 4th International Workshop on Conducting Empirical Studies in Industry (CESI), pages 1–3.
Juristo, N. and Gómez, O. S. (2012). Replication of Software Engineering Experiments, pages 60–88. Springer Berlin Heidelberg, Berlin, Heidelberg.
Karac, I., Panach, J. I., Turhan, B., and Juristo, N. (2025). Does treatment adherence impact experiment results in tdd? IEEE Transactions on Software Engineering, 51(1):135–152.
Kittur, A., Chi, E. H., and Suh, B. (2008). Crowdsourcing user studies with mechanical turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 453–456.
Kogut, T. and Ritov, I. (2005). The ”identified victim” effect: An identified group, or just a single individual? Journal of Behavioral Decision Making, 18(3):157–167.
Kollanus, S. (2010). Test-driven development: still a promising approach? Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pages 1–4.
Madeyski, L. (2010). The impact of test-first programming on branch coverage and mutation score indicator of unit tests: An experiment. Information and Software Technology, 52(2):169–184.
McGonigal, J. (2011). Reality Is Broken: Why Games Make Us Better and How They Can Change the World. Penguin Press, New York.
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67(4):371–378.
Mischel, W., Shoda, Y., and Rodriguez, M. L. (1989). Delay of gratification in children. Science, 244(4907):933–938.
Munir, H., Moayyed, M., and Petersen, K. (2014). Considering rigor and relevance when evaluating test driven development: A systematic review. Information and Software Technology, 56(4):375–394.
Nagappan, N., Maximilien, E. M., Bhat, T., and Williams, L. (2008). Realizing quality improvement through test driven development: Results and experiences of four industrial teams. Empirical Software Engineering, 13(3):289–302.
Oppenheimer, D. M., Meyvis, T., and Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4):867–872.
Pančur, M. and Ciglarič, M. (2011). Impact of test-driven development on productivity, code and tests: A controlled experiment. Information and Software Technology, 53(6):557–573.
Rafique, Y. and Mišić, V. B. (2012). Effects of test driven development on external quality and productivity: A systematic review. Empirical Software Engineering, 18(5):1094– 1117.
Raura Ruiz, J. G. (2022). Impacto de las características personales de los programadores en la efectividad de Test-Driven Development (TDD). PhD thesis, Universidad Nacional de La Plata, La Plata, Argentina. Tesis de doctorado.
Santos, A., Vegas, S., Dieste, O., and Juristo, N. (2021). A family of experiments on test-driven development. Empirical Software Engineering, 26(42).
Senn, S. (2003). Cross-over Trials in Clinical Research. Statistics in Practice. Wiley.
Shadish, W., Cook, T., and Campbell, D. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2):257–285.
Vegas, S., Apa, C., and Juristo, N. (2016). Crossover designs in software engineering experiments: Benefits and perils. IEEE Transactions on Software Engineering, 42(2):120–135.
Zhou, X. and Fishbach, A. (2016). The pitfall of experimenting on the web: How unat-tended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4):493–504.
Bissi, W., Neto, A. G. S. S., and Emer, M. C. F. P. (2016a). The effects of test driven development on internal quality, external quality and productivity: A systematic review. Information and Software Technology, 74:45–54.
Bissi, W., Scanniello, G., Romano, S., and Tortora, G. (2016b). On the perception of test driven development: A replicated study. Journal of Visual Languages & Computing, 34:11–24.
Chandler, J., Mueller, P., and Paolacci, G. (2014). Nonnaïveté among amazon mechanical turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1):112–130.
Deci, E. L. and Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Plenum, New York.
Diefendorff, J. M. and Richard, E. M. (2000). Antecedents and consequences of emotional display rule perceptions. Journal of Applied Psychology, 88(2):284–294.
Eysenck, M. W., Derakshan, N., Santos, R., and Calvo, M. G. (2007). Anxiety and cognitive performance: Attentional control theory. Emotion, 7(2):336–353.
Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., and Juristo, N. (2016). A dissection of the test-driven development process: does it really matter to test-first or to test-last? IEEE Transactions on Software Engineering, 43(7):597–614.
Fucci, D., Turhan, B., and Oivo, M. (2014). Impact of process conformance on the effects of test-driven development. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pages 1–10.
George, B. and Williams, L. (2004). A structured experiment of test-driven development. Information and Software Technology, 46(5):337–342.
Hauser, D. J. and Schwarz, N. (2018). Attentive turkers: Mturk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1):400–407.
Juristo, N. (2016). Experiences conducting experiments in industry: The eseil fidipro project. In 2016 IEEE/ACM 4th International Workshop on Conducting Empirical Studies in Industry (CESI), pages 1–3.
Juristo, N. and Gómez, O. S. (2012). Replication of Software Engineering Experiments, pages 60–88. Springer Berlin Heidelberg, Berlin, Heidelberg.
Karac, I., Panach, J. I., Turhan, B., and Juristo, N. (2025). Does treatment adherence impact experiment results in tdd? IEEE Transactions on Software Engineering, 51(1):135–152.
Kittur, A., Chi, E. H., and Suh, B. (2008). Crowdsourcing user studies with mechanical turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 453–456.
Kogut, T. and Ritov, I. (2005). The ”identified victim” effect: An identified group, or just a single individual? Journal of Behavioral Decision Making, 18(3):157–167.
Kollanus, S. (2010). Test-driven development: still a promising approach? Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pages 1–4.
Madeyski, L. (2010). The impact of test-first programming on branch coverage and mutation score indicator of unit tests: An experiment. Information and Software Technology, 52(2):169–184.
McGonigal, J. (2011). Reality Is Broken: Why Games Make Us Better and How They Can Change the World. Penguin Press, New York.
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67(4):371–378.
Mischel, W., Shoda, Y., and Rodriguez, M. L. (1989). Delay of gratification in children. Science, 244(4907):933–938.
Munir, H., Moayyed, M., and Petersen, K. (2014). Considering rigor and relevance when evaluating test driven development: A systematic review. Information and Software Technology, 56(4):375–394.
Nagappan, N., Maximilien, E. M., Bhat, T., and Williams, L. (2008). Realizing quality improvement through test driven development: Results and experiences of four industrial teams. Empirical Software Engineering, 13(3):289–302.
Oppenheimer, D. M., Meyvis, T., and Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4):867–872.
Pančur, M. and Ciglarič, M. (2011). Impact of test-driven development on productivity, code and tests: A controlled experiment. Information and Software Technology, 53(6):557–573.
Rafique, Y. and Mišić, V. B. (2012). Effects of test driven development on external quality and productivity: A systematic review. Empirical Software Engineering, 18(5):1094– 1117.
Raura Ruiz, J. G. (2022). Impacto de las características personales de los programadores en la efectividad de Test-Driven Development (TDD). PhD thesis, Universidad Nacional de La Plata, La Plata, Argentina. Tesis de doctorado.
Santos, A., Vegas, S., Dieste, O., and Juristo, N. (2021). A family of experiments on test-driven development. Empirical Software Engineering, 26(42).
Senn, S. (2003). Cross-over Trials in Clinical Research. Statistics in Practice. Wiley.
Shadish, W., Cook, T., and Campbell, D. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2):257–285.
Vegas, S., Apa, C., and Juristo, N. (2016). Crossover designs in software engineering experiments: Benefits and perils. IEEE Transactions on Software Engineering, 42(2):120–135.
Zhou, X. and Fishbach, A. (2016). The pitfall of experimenting on the web: How unat-tended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4):493–504.
Publicado
12/05/2025
Como Citar
RAURA, Geovanny; FONSECA C., Efraín R.; DIESTE, Oscar.
Factores Personales que Influyen en la Realización de Tareas bajo Test-Driven Development: Meta-Análisis de una Familia de Experimentos. In: CONGRESSO IBERO-AMERICANO EM ENGENHARIA DE SOFTWARE (CIBSE), 28. , 2025, Ciudad Real/Espanha.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 165-179.
DOI: https://doi.org/10.5753/cibse.2025.35300.
