Refactoring Assertion Roulette and Duplicate Assert test smells: a controlled experiment

• Railana Santana UFBA http://orcid.org/0000-0002-1153-8960
• Luana Martins UFBA http://orcid.org/0000-0001-6340-7615
• Tássio Virgínio IFTO http://orcid.org/0000-0001-6259-4957
• Larissa Soares UFBA / UEFS http://orcid.org/0000-0002-8069-5249
• Heitor Costa UFLA http://orcid.org/0000-0002-9903-7414
• Ivan Machado UFBA http://orcid.org/0000-0001-9027-2293

DOI: https://doi.org/10.5753/cibse.2022.20977

Resumo

Test smells can reduce the developers' ability to interact with the test code. Refactoring test code offers a safe strategy to handle test smells. However, the manual refactoring activity is not a trivial process, and it is often tedious and error-prone. This study aims to evaluate RAIDE, a tool for automatic identification and refactoring of test smells. We present an empirical assessment of RAIDE, in which we analyzed its capability at refactoring Assertion Roulette and Duplicate Assert test smells and compared the results against both manual refactoring and a state-of-the-art approach. The results show that RAIDE provides a faster and more intuitive approach for handling test smells than using an automated tool for smells detection combined with manual refactoring.

Palavras-chave: Test Smells, Unit Tests, Controlled Experiment

Referências

Aljedaani, W., Peruma, A., Aljohani, A., Alotaibi, M., Mkaouer, M. W., Ouni, A., Newman, C. D., Ghallab, A., and Ludi, S. (2021). Test smell detection tools: A systematic mapping study. In Evaluation and Assessment in Software Engineering, page 170–180, New York, NY, USA. ACM.

Baker, P., Evans, D., Grabowski, J., Neukirchen, H., and Zeiss, B. (2006). Trex-the refactoring and metrics tool for ttcn-3 test specifications. In Testing: Academic & Industrial Conference-Practice And Research Techniques, pages 90–94, UK. IEEE.

Garousi, V. and Kuçük, B. (2018). Smells in software test code: A survey of knowledge in industry and academia. Journal of systems and software, 138:52–81.

Greiler, M., van Deursen, A., and Storey, M.-A. (2013a). Automated detection of test fixture strategies and smells. In 60th International Conference on Software Testing, Verification and Validation, pages 322–331, Luxembourg. IEEE.

Greiler, M., Zaidman, A., Deursen, A. v., and Storey, M.-A. (2013b). Strategies for avoiding text fixture smells during software evolution. In Proc. of the 10th Working Conference on Mining Software Repositories, pages 387–396, San Francisco, CA, USA. IEEE Press.

Kim, D. J., Chen, T.-H. P., and Yang, J. (2021). The secret life of test smells-an empirical study on test smell evolution and maintenance. Empirical Software Engineering, 26(5):1–47.

Lambiase, S., Cupito, A., Pecorelli, F., De Lucia, A., and Palomba, F. (2020). Justin-time test smell detection and refactoring: The darts project. In Proc. of the 28th International Conference on Program Comprehension, page 441–445, New York, NY, USA. ACM.

Mann, H. B. and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. In The annals of mathematical statistics, volume 18, pages 50–60, Michigan. JSTOR.

Martinez, M., Etien, A., Ducasse, S., and Fuhrman, C. (2020). Rtj: A java framework for detecting and refactoring rotten green test cases. In Proc. of the 42nd International Conference on Software Engineering: Companion Proceedings, page 69–72, New York, NY, USA. ACM.

Palomba, F., Di Nucci, D., Panichella, A., Oliveto, R., and De Lucia, A. (2016). On the diffusion of test smells in automatically generated test code: An empirical study. In Proc. of the 9th international workshop on search-based software testing, pages 5–14, Austin, TX, USA. ACM, IEEE.

Palomba, F., Zaidman, A., and De Lucia, A. (2018). Automatic test smell detection using information retrieval techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 311–322, Madrid, Spain. IEEE.

Peruma, A., Almalki, K., Newman, C. D., Mkaouer, M. W., Ouni, A., and Palomba, F. (2019). On the distribution of test smells in open source android applications: An exploratory study. In Proc. of the 29th Annual International Conference on Computer Science and Software Engineering, page 193–202, USA. IBM Corp.

Peruma, A. S. A. (2018). What the Smell? An Empirical Investigation on the Distribution and Severity of Test Smells in Open Source Android Applications. Ph.d. thesis, Rochester Institute of Technology, Rochester, New York.

Santana, R., Martins, L., Rocha, L., Virgínio, T., Cruz, A., Costa, H., and Machado, I. (2020). RAIDE: a tool for assertion roulette and duplicate assert identification and refactoring. In Proc. of the XXXIV Brazilian Symposium on Software Engineering - Tools Track, pages 374–379, NY, USA. ACM.

Van Deursen, A., Moonen, L., Van Den Bergh, A., and Kok, G. (2001). Refactoring test code. In Proc. of the 2nd international conference on extreme programming and flexible processes in software engineering (XP2001), pages 92–95, NLD. CWI (Centre for Mathematics and Computer Science).

Vegas, S., Apa, C., and Juristo, N. (2015). Crossover designs in software engineering experiments: Benefits and perils. IEEE Transactions on Software Engineering, 42(2):120–135.

Virgínio, T., Martins, L., Rocha, L., Santana, R., Cruz, A., Costa, H., and Machado, I. (2020a). Jnose: Java test smell detector. In Proc. of the 34th Brazilian Symposium on Software Engineering, SBES ’20, page 564–569, New York, NY, USA. Association for Computing Machinery.

Virgínio, T., Martins, L. A., Soares, L. R., Santana, R., Costa, H., and Machado, I. (2020b). An empirical study of automatically-generated tests from the perspective of test smells. In Proc. of the 34th Brazilian Symposium on Software Engineering, page 92–96, New York, NY, USA. ACM.

Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., and Regnell, B. (2012). Experimentation in Software Engineering. Springer.

Yusifoglu, V. G., Amannejad, Y., and Can, A. B. (2015). Software test-code engineering: A systematic mapping. Information and Software Technology, 58:123–147.