Automated Behavioral Testing of Refactoring Engines
Resumo
Refactoring is a transformation that preserves the external behavior of a program and improves its internal quality. To help developers in this task, current IDEs, such as Eclipse and NetBeans, automate a number of refactorings. However, implementing refactorings is a complex task, and so, even mainstream IDEs contain critical bugs. We propose an automated approach for testing of Java refactoring engines. Its key components are: JDOLLY, a Java program generator, and SAFEREFACTOR, a program for checking behavioral changes. The technique uses JDOLLY to generate programs as test inputs. For each generated program, it applies the refactoring by using the engine under test, and uses oracles based on SAFEREFACTOR to evaluate the correctness of the transformation. In the end, it classifies the failures into distinct bugs. We have evaluated this technique by testing up to 10 refactorings from Eclipse, NetBeans and the JastAdd Refactoring Tools. Our technique tested 153,444 transformations, and identified more than 100 bugs, which were reported to engines’ developers. They accepted most of them, and already fixed 35 bugs.
Referências
Gligoric, M., Gvero, T., Jagannath, V., Khurshid, S., Kuncak, V., and Marinov, D. (2010). Test generation through programming in udita. In ICSE ’10, pages 225–234.
Jackson, D. (2012). Software Abstractions: Logic, Language, and Analysis. The MIT Press.
Jagannath, V., Lee, Y. Y., Daniel, B., and Marinov, D. (2009). Reducing the costs of bounded-exhaustive testing. In FASE ’09, pages 171–185.
Opdyke, W. (1992). Refactoring Object-Oriented Frameworks. PhD thesis, University of Illinois at Urbana-Champaign.
Pacheco, C., Lahiri, S., Ernst, M., and Ball, T. (2007). Feedback-directed random test generation. In ICSE ’07, pages 75–84.
Schäfer, M. and de Moor, O. (2010). Specifying and implementing refactorings. In OOPSLA ’10, pages 286–301. ACM.
Schäfer, M., Ekman, T., and de Moor, O. Challenge proposal: verification of refactorings. In PLPV ’09, pages 67–72. ACM.
Soares, G., Cat˜ao, B., Varj˜ao, C., Aguiar, S., Gheyi, R., and Massoni, T. (2011a). Analyzing refactorings on software repositories. In SBES ’11, pages 164–173.
Soares, G., Cavalcanti, D., and Gheyi, R. (2011b). Making aspect-oriented refactoring safer. In SBLP’11.
Soares, G., Gheyi, R., and Massoni, T. (2013a). Automated behavioral testing of refactoring engines. IEEE TSE, 39(2):147–162.
Soares, G., Gheyi, R., Murphy-Hill, E., and Johnson, B. (2013b). Comparing approaches to analyze refactoring activity on software repositories. JSS, 86:1006,1022.
Soares, G., Gheyi, R., Serey, D., and Massoni, T. (2010). Making program refactoring safer. IEEE Software, 27:52–57.
Soares, G., Mongiovi, M., and Gheyi, R. (2011c). Identifying overly strong conditions in refactoring implementations. In ICSM ’11, pages 173–182.
