TAPN: Test Automation’s Pyramid of Needs
Resumo
Testing is an essential activity to ensure quality of software systems, but it is expensive and time consuming. Thus, testing automation would be an alternative to improve test productivity and save costs. However, many organizations refuse to use test automation or had failed on implement it because they do not know how to deal with the implementation of a test automation strategy fitted to their goals and expectations. Most of them underestimate or have no knowledge about test automation factor of success. In addition, although there are many works and maturity models focused on improving the testing process, few ones focus on test automation issues. The main contribution of this paper is to propose a hierarchical model called Test Automation’s Pyramid of Needs (TAPN). TAPN is inspired in the Maslow’s Pyramid of Needs administration theory and it comprises five levels that influence on the success of test automation initiatives in software organizations. TAPN intends to help organizations to build their test automation strategy using good practices in test automation captured from the technical literature.
Palavras-chave:
Test, Pyramid of Needs, Automation’s
Referências
Alam, M. N., “Software test automation myths and facts”, IMI Systems Inc., Dallas, 2007.
Anand, S. et al. “An orchestrated survey of methodologies for automated software test case generation”. J. Syst. Softw. 86, 8, 2013, pp. 1978-2001.
Andersin, J., “TPI – a model for Test Process Improvement”, Department of Computer Science, University of Helsink, Helsinki, 2004.
Bach, J., “Test Automation Snake Oil”, In 14th International Conference and Exposition on Testing Computer Software, Washington-DC, 1997, p. 6.
Bansal, A., Muli, M., Patil, K., “Taming complexity while gaining efficiency: Requirements for the next generation of test automation tools”, In: AUTOTESTCON, pp. 1-6, 2013.
Bertolino, A., “An overview of automated software testing”, Journal of Systems and Software, vol. 15, no. 2, pp. 133-138, 1991.
Brunstein, I., Suwanassart, T., Carlson, R., “Developing a testing maturity model for software test process evaluation and improvement”, In International Test Conference, 1996, pp. 581-589.
Cervantes, A., “Exploring the use of a test automation framework”, In Aerospace conference, 2009 IEEE, 2009, pp. 1-9.
Eldth, S., Andersson, K., Wirlund, K., “Towards a Test Automation Improvement Model (TAIM)”, in International Conference on Software Testing. Verification and Validation Workshops, 2014, pp. 337-342.
Graham, D., Fewster, M., “Experiences of Test Automation: Case Studies of Software Test Automation”, Addison-Wesley Professional, 2012.
Hayes, L. G., “The Automated Testing Handbook”. Software Testing Institute, 1995.
Heiskanen, H., Maunumaa, M. and Katara, M.Test Process Improvement for Automated Test Generation. Tampere: Tampere University of Technology, Department of Software Systems, 2010.
Hoffman, D., “Cost benefits analysis of test automation”. STAR West, v. 99, 1999.
IQUIP, “TMap”, TMap Sogeti Nederland B.V., 2004, Available at: http://www.tmap.net. Accessed at March 10th, 2015.
Höhn, E. N., KITest: A Framework of software testing process knowledge and improvement (in Portuguese), Dissertation presented in Institute of Mathematical and Computing Sciences ICMC-USP, July, 2011.
Karhu K., Repo T., Taipale, O., Smolander K. 2009. “Empirical Observations on Software Testing Automation”. In Proceedings of the 2009 International Conference on Software Testing Verification and Validation (ICST'09). IEEE Computer Society, Washington, DC, USA, pp. 201-209.
Kasurinen, J., Taipale, O., Smolander, K., “Software test automation in practice: empirical observations”, Advances in Software Engineering, 2010.
Kim, E. H., Na J. C., Ryoo S. M., "Implementing an effective test automation framework," in Computer Software and Applications Conference, 2009. COMPSAC'09. 33rd Annual IEEE International, 2009, pp. 534-538.
Kulkarni, S., “Test Process Maturity Models–Yesterday, Today and Tomorrow”, in Proceedings of the 6th Annual International Software Testing Conference, Delhi, 2006.
Lee, J., Kang, S., Lee D., “Survey on software testing practices”, IET software, vol. 6, no. 3, pp. 275-282, 2012.
Lott, M., Fecko, C.M., “Lessons learned from automating tests for an operations support system”, In: Software: Practice and Experience, pp. 1-23, 2002.
Maslow, A. H., “A theory of human motivation”, Psychological review, vol. 50, no. 4, p. 370, 1943.
Meng, X. F., “Analysis of software automation test protocol”, In Electronic and Mechanical Engineering and Information Technology (EMEIT), 2011 International Conference on, 2011, pp. 4138-4141.
MPT.Br, “Melhoria do Processo de Teste Brasileiro MPT.br”, SOFTEX RECIFE, 2011.
Obele, B. O., Kim, D., “On an Embedded Software Design Architecture for Improving the Testability of In-vehicle Multimedia Software”, In IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2014, pp. 349-352.
Oliveira, J. C., Gouveia, C. C., Quidute-Filho, R., “Test Automation Viability Analysis Method”, In Proceedings of the 7th IEEE Latin American Test Workshop (LATW’2006), 2006, p. 6.
Pettichord, B., “Seven steps to test automation success”, Proceedings of the International Conference on Software Testing, Analysis, and Review, 1999.
Rafi, D. M., Moses, K. R. K., Peterson, K., Mäntylä, M. V. “Benefits and limitations of automated software testing: systematic literature review and practitioner survey”. In Proceedings of the 7th International Workshop on Automation of Software Test (AST '12). IEEE Press, Piscataway, NJ, USA, 2012, pp. 36-42.
Ramler, R., Wolfmaier, K., “Economic perspectives in test automation: balancing automated and manual testing with opportunity cost”, In Proceedings of the 2006 international workshop on Automation of software test, 2006, pp. 85-91.
Rankin, C., “The Software Testing Automation Framework”, IBM Systems Journal, vol. 41, no. 1, pp. 126-139, 2002.
Song, H., Ryoo, S., Kim, J. H., “An Integrated Test Automation Framework for Testing on Heterogeneous Mobile Platforms”, In ACIS International Symposium on Software and Network Engineering (SSNE), 2011, pp. 141-145.
Taipale, O., Kasurinen, J., Karhu, K., Smolander, K., “Trade-off between automated and manual software testing”, International Journal of System Assurance Engineering and Management, vol. 2, no. 2, pp. 114-125, 2011.
Veenendaal, E. Van, “Test Maturity Model integration (TMMi)”, TMMi Foundation,2010.
Young , D., “Test Automation: An architectured approach”, Schwab Performance Technologies, Raleigh, 2004.
Anand, S. et al. “An orchestrated survey of methodologies for automated software test case generation”. J. Syst. Softw. 86, 8, 2013, pp. 1978-2001.
Andersin, J., “TPI – a model for Test Process Improvement”, Department of Computer Science, University of Helsink, Helsinki, 2004.
Bach, J., “Test Automation Snake Oil”, In 14th International Conference and Exposition on Testing Computer Software, Washington-DC, 1997, p. 6.
Bansal, A., Muli, M., Patil, K., “Taming complexity while gaining efficiency: Requirements for the next generation of test automation tools”, In: AUTOTESTCON, pp. 1-6, 2013.
Bertolino, A., “An overview of automated software testing”, Journal of Systems and Software, vol. 15, no. 2, pp. 133-138, 1991.
Brunstein, I., Suwanassart, T., Carlson, R., “Developing a testing maturity model for software test process evaluation and improvement”, In International Test Conference, 1996, pp. 581-589.
Cervantes, A., “Exploring the use of a test automation framework”, In Aerospace conference, 2009 IEEE, 2009, pp. 1-9.
Eldth, S., Andersson, K., Wirlund, K., “Towards a Test Automation Improvement Model (TAIM)”, in International Conference on Software Testing. Verification and Validation Workshops, 2014, pp. 337-342.
Graham, D., Fewster, M., “Experiences of Test Automation: Case Studies of Software Test Automation”, Addison-Wesley Professional, 2012.
Hayes, L. G., “The Automated Testing Handbook”. Software Testing Institute, 1995.
Heiskanen, H., Maunumaa, M. and Katara, M.Test Process Improvement for Automated Test Generation. Tampere: Tampere University of Technology, Department of Software Systems, 2010.
Hoffman, D., “Cost benefits analysis of test automation”. STAR West, v. 99, 1999.
IQUIP, “TMap”, TMap Sogeti Nederland B.V., 2004, Available at: http://www.tmap.net. Accessed at March 10th, 2015.
Höhn, E. N., KITest: A Framework of software testing process knowledge and improvement (in Portuguese), Dissertation presented in Institute of Mathematical and Computing Sciences ICMC-USP, July, 2011.
Karhu K., Repo T., Taipale, O., Smolander K. 2009. “Empirical Observations on Software Testing Automation”. In Proceedings of the 2009 International Conference on Software Testing Verification and Validation (ICST'09). IEEE Computer Society, Washington, DC, USA, pp. 201-209.
Kasurinen, J., Taipale, O., Smolander, K., “Software test automation in practice: empirical observations”, Advances in Software Engineering, 2010.
Kim, E. H., Na J. C., Ryoo S. M., "Implementing an effective test automation framework," in Computer Software and Applications Conference, 2009. COMPSAC'09. 33rd Annual IEEE International, 2009, pp. 534-538.
Kulkarni, S., “Test Process Maturity Models–Yesterday, Today and Tomorrow”, in Proceedings of the 6th Annual International Software Testing Conference, Delhi, 2006.
Lee, J., Kang, S., Lee D., “Survey on software testing practices”, IET software, vol. 6, no. 3, pp. 275-282, 2012.
Lott, M., Fecko, C.M., “Lessons learned from automating tests for an operations support system”, In: Software: Practice and Experience, pp. 1-23, 2002.
Maslow, A. H., “A theory of human motivation”, Psychological review, vol. 50, no. 4, p. 370, 1943.
Meng, X. F., “Analysis of software automation test protocol”, In Electronic and Mechanical Engineering and Information Technology (EMEIT), 2011 International Conference on, 2011, pp. 4138-4141.
MPT.Br, “Melhoria do Processo de Teste Brasileiro MPT.br”, SOFTEX RECIFE, 2011.
Obele, B. O., Kim, D., “On an Embedded Software Design Architecture for Improving the Testability of In-vehicle Multimedia Software”, In IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2014, pp. 349-352.
Oliveira, J. C., Gouveia, C. C., Quidute-Filho, R., “Test Automation Viability Analysis Method”, In Proceedings of the 7th IEEE Latin American Test Workshop (LATW’2006), 2006, p. 6.
Pettichord, B., “Seven steps to test automation success”, Proceedings of the International Conference on Software Testing, Analysis, and Review, 1999.
Rafi, D. M., Moses, K. R. K., Peterson, K., Mäntylä, M. V. “Benefits and limitations of automated software testing: systematic literature review and practitioner survey”. In Proceedings of the 7th International Workshop on Automation of Software Test (AST '12). IEEE Press, Piscataway, NJ, USA, 2012, pp. 36-42.
Ramler, R., Wolfmaier, K., “Economic perspectives in test automation: balancing automated and manual testing with opportunity cost”, In Proceedings of the 2006 international workshop on Automation of software test, 2006, pp. 85-91.
Rankin, C., “The Software Testing Automation Framework”, IBM Systems Journal, vol. 41, no. 1, pp. 126-139, 2002.
Song, H., Ryoo, S., Kim, J. H., “An Integrated Test Automation Framework for Testing on Heterogeneous Mobile Platforms”, In ACIS International Symposium on Software and Network Engineering (SSNE), 2011, pp. 141-145.
Taipale, O., Kasurinen, J., Karhu, K., Smolander, K., “Trade-off between automated and manual software testing”, International Journal of System Assurance Engineering and Management, vol. 2, no. 2, pp. 114-125, 2011.
Veenendaal, E. Van, “Test Maturity Model integration (TMMi)”, TMMi Foundation,2010.
Young , D., “Test Automation: An architectured approach”, Schwab Performance Technologies, Raleigh, 2004.
Publicado
17/08/2015
Como Citar
RODRIGUES, Anderson; DIAS-NETO, Arilo C.; BEZERRA, Allan.
TAPN: Test Automation’s Pyramid of Needs. In: SIMPÓSIO BRASILEIRO DE QUALIDADE DE SOFTWARE (SBQS), 14. , 2015, Manaus.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2015
.
p. 122-136.
DOI: https://doi.org/10.5753/sbqs.2015.15218.
