Feature-Trace: Generating Operational Profile and Supporting Testing Prioritization from BDD Features
Resumo
Operational Profiles provide quantitative information about how the software will be used, which supports highlighting those software components more sensitive to reliability based on their profile usage. However, the generation of Operational Profiles usually requires a considerable team effort to liaise requirements specification until their reification into expected software artifacts. In this sense, it becomes paramount in the software life cycle the ability to seamlessly or efficiently perform traceability from requirement to code, embracing the testing process as a means to ensure that the requirements are satisfiably covered and addressed. In this work, we propose the Feature-Trace approach which merges the advantages of the Operational Profile and the benefits of the requirements-to-code traceability present in the BDD (Behavior-Driven Development) approach. The primary goal of our work is to use the BDD approach as an information source for the semi-automated generation of the Operational Profile. The proposed approach was evaluated on the Diaspora software, on a GitHub open source software. The case study revealed that the Feature-Trace approach is capable of extracting the operational profile seamlessly from the specified Diaspora's BDD features as well as obtaining and presenting vital information to guide the process of test cases prioritization.
Palavras-chave:
operational profile, behavior-driven development, requirements traceability, testing
Referências
Fabian Beck, Bogdan Dit, Jaleo Velasco-Madden, Daniel Weiskopf, and Denys Poshyvanyk. 2015. Rethinking user interfaces for feature location. In Proceedings of the 2015 IEEE 23rd Intl. Conf. on Program Comprehension. IEEE Press, 151--162.
Kent Beck. 2003. Test-driven development: by example. Addison-Wesley Professional.
Antonia Bertolino, Breno Miranda, Roberto Pietrantuono, and Stefano Russo. 2017. Adaptive coverage and operational profile-based testing for reliability improvement. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 541--551.
Elizabeth Bjarnason, Per Runeson, Markus Borg, Michael Unterkalmsteiner, Emelie Engström, Björn Regnell, Giedre Sabaliauskaite, Annabella Loconsole, Tony Gorschek, and Robert Feldt. 2014. Challenges and practices in aligning requirements with verification and validation: a case study of six companies. Empirical Software Engineering 19, 6 (2014), 1809--1855.
Cagatay Catal and Deepti Mishra. 2013. Test case prioritization: a systematic mapping study. Software Quality Journal 21, 3 (2013), 445--478.
Bogdan Dit, Meghan Revelle, Malcom Gethers, and Denys Poshyvanyk. 2013. Feature location in source code: a taxonomy and survey. Journal of software: Evolution and Process 25, 1 (2013), 53--95.
Marc Eaddy, Alfred V Aho, Giuliano Antoniol, and Yann-Gaël Guéhéneuc. 2008. Cerberus: Tracing requirements to source code using information retrieval, dynamic analysis, and program analysis. In 2008 16th IEEE Intl. Conf. on Program Comprehension. Ieee, 53--62.
Christof Ebert and James Cain. 2016. Cyclomatic complexity. IEEE software 33, 6 (2016), 27--29.
Thomas Eisenbarth, Rainer Koschke, and Daniel Simon. 2003. Locating features in source code. IEEE Transactions on software engineering 29, 3 (2003), 210--224.
Marcelo Medeiros Eler, Andre Takeshi Endo, Paulo Cesar Masiero, Marcio Eduardo Delamaro, Jose Carlos Maldonado, Auri Marcelo Rizzo Vincenzi, Marcos Lordello Chaim, and Delano Medeiros Beder. 2009. JaBUTiService: a web service for structural testing of java programs. In 2009 33rd Annual IEEE Software Engineering Workshop. IEEE, 69--76.
Eric Evans. 2004. Domain-driven design: tackling complexity in the heart of software. Addison-Wesley Professional.
Jerry Fitzpatrick. 1997. Applying the ABC metric to C, C++, and Java. More C++ gems (1997), 245--264.
Ahmed E Hassan, Daryl J Martin, Parminder Flora, Paul Mansfield, and Dave Dietz. 2008. An industrial case study of customizing operational profiles using log compression. In Proceedings of the 30th international conference on Software engineering. ACM, 713--723.
Christopher Henard, Mike Papadakis, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Comparing white-box and black-box test prioritization. In Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 523--534.
Yi Li, Chenguang Zhu, Julia Rubin, and Marsha Chechik. 2017. FHistorian: Locating features in version histories. In Proceedings of the 21st International Systems and Software Product Line Conference-Volume A. ACM, 49--58.
Dapeng Liu, Andrian Marcus, Denys Poshyvanyk, and Vaclav Rajlich. 2007. Feature location via information retrieval based filtering of a single scenario execution trace. In Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering. ACM, 234--243.
Thomas J McCabe. 1976. A complexity measure. IEEE Transactions on software Engineering 4 (1976), 308--320.
Breno Miranda, Emilio Cruciani, Roberto Verdecchia, and Antonia Bertolino. 2018. Fast approaches to scalable similarity-based test case prioritization. In Proceedings of the 40th International Conference on Software Engineering. ACM, 222--232.
John D Musa. 1993. Operational profiles in software-reliability engineering. IEEE software 2 (1993), 14--32.
John D Musa. 1996. The operational profile. In Reliability and Maintenance of Complex Systems. Springer, 333--344.
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons, New Jersey, USA.
Meiyappan Nagappan, Kesheng Wu, and Mladen A Vouk. 2009. Efficiently extracting operational profiles from execution logs using suffix arrays. In Software Reliability Engineering, 2009. ISSRE'09. 20th International Symposium on. IEEE, 41--50.
Tanzeem Bin Noor and Hadi Hemmati. 2015. A similarity-based approach for test case prioritization using historical failure data. In 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 58--68.
Dan North. 2003. Introducing BDD. https://dannorth.net/introducing-bdd/
Joao Felipe S Ouriques, Emanuela G Cartaxo, and Patrícia DL Machado. 2018. Test case prioritization techniques for model-based testing: a replicated study. Software Quality Journal (2018), 1--32.
Patrick Rempel and Parick Mäder. 2017. Preventing defects: The impact of requirements traceability completeness on software quality. IEEE Transactions on Software Engineering 43, 8 (2017), 777--797.
Per Runeson and Björn Regnell. 1998. Derivation of an integrated operational profile and use case model. In Software Reliability Engineering, 1998. Proceedings. The Ninth International Symposium on. IEEE, 70--79.
John Ferguson Smart. 2014. BDD in Action. Manning Publications.
Hany EL Yamany and Miriam AM Capretz. 2007. A Multi-Agent Framework for Building an Automatic Operational Profile. In Advances and Innovations in Systems, Computing Sciences and Software Engineering. Springer, 161--166.
Kent Beck. 2003. Test-driven development: by example. Addison-Wesley Professional.
Antonia Bertolino, Breno Miranda, Roberto Pietrantuono, and Stefano Russo. 2017. Adaptive coverage and operational profile-based testing for reliability improvement. In Proceedings of the 39th International Conference on Software Engineering. IEEE Press, 541--551.
Elizabeth Bjarnason, Per Runeson, Markus Borg, Michael Unterkalmsteiner, Emelie Engström, Björn Regnell, Giedre Sabaliauskaite, Annabella Loconsole, Tony Gorschek, and Robert Feldt. 2014. Challenges and practices in aligning requirements with verification and validation: a case study of six companies. Empirical Software Engineering 19, 6 (2014), 1809--1855.
Cagatay Catal and Deepti Mishra. 2013. Test case prioritization: a systematic mapping study. Software Quality Journal 21, 3 (2013), 445--478.
Bogdan Dit, Meghan Revelle, Malcom Gethers, and Denys Poshyvanyk. 2013. Feature location in source code: a taxonomy and survey. Journal of software: Evolution and Process 25, 1 (2013), 53--95.
Marc Eaddy, Alfred V Aho, Giuliano Antoniol, and Yann-Gaël Guéhéneuc. 2008. Cerberus: Tracing requirements to source code using information retrieval, dynamic analysis, and program analysis. In 2008 16th IEEE Intl. Conf. on Program Comprehension. Ieee, 53--62.
Christof Ebert and James Cain. 2016. Cyclomatic complexity. IEEE software 33, 6 (2016), 27--29.
Thomas Eisenbarth, Rainer Koschke, and Daniel Simon. 2003. Locating features in source code. IEEE Transactions on software engineering 29, 3 (2003), 210--224.
Marcelo Medeiros Eler, Andre Takeshi Endo, Paulo Cesar Masiero, Marcio Eduardo Delamaro, Jose Carlos Maldonado, Auri Marcelo Rizzo Vincenzi, Marcos Lordello Chaim, and Delano Medeiros Beder. 2009. JaBUTiService: a web service for structural testing of java programs. In 2009 33rd Annual IEEE Software Engineering Workshop. IEEE, 69--76.
Eric Evans. 2004. Domain-driven design: tackling complexity in the heart of software. Addison-Wesley Professional.
Jerry Fitzpatrick. 1997. Applying the ABC metric to C, C++, and Java. More C++ gems (1997), 245--264.
Ahmed E Hassan, Daryl J Martin, Parminder Flora, Paul Mansfield, and Dave Dietz. 2008. An industrial case study of customizing operational profiles using log compression. In Proceedings of the 30th international conference on Software engineering. ACM, 713--723.
Christopher Henard, Mike Papadakis, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Comparing white-box and black-box test prioritization. In Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 523--534.
Yi Li, Chenguang Zhu, Julia Rubin, and Marsha Chechik. 2017. FHistorian: Locating features in version histories. In Proceedings of the 21st International Systems and Software Product Line Conference-Volume A. ACM, 49--58.
Dapeng Liu, Andrian Marcus, Denys Poshyvanyk, and Vaclav Rajlich. 2007. Feature location via information retrieval based filtering of a single scenario execution trace. In Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering. ACM, 234--243.
Thomas J McCabe. 1976. A complexity measure. IEEE Transactions on software Engineering 4 (1976), 308--320.
Breno Miranda, Emilio Cruciani, Roberto Verdecchia, and Antonia Bertolino. 2018. Fast approaches to scalable similarity-based test case prioritization. In Proceedings of the 40th International Conference on Software Engineering. ACM, 222--232.
John D Musa. 1993. Operational profiles in software-reliability engineering. IEEE software 2 (1993), 14--32.
John D Musa. 1996. The operational profile. In Reliability and Maintenance of Complex Systems. Springer, 333--344.
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons, New Jersey, USA.
Meiyappan Nagappan, Kesheng Wu, and Mladen A Vouk. 2009. Efficiently extracting operational profiles from execution logs using suffix arrays. In Software Reliability Engineering, 2009. ISSRE'09. 20th International Symposium on. IEEE, 41--50.
Tanzeem Bin Noor and Hadi Hemmati. 2015. A similarity-based approach for test case prioritization using historical failure data. In 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 58--68.
Dan North. 2003. Introducing BDD. https://dannorth.net/introducing-bdd/
Joao Felipe S Ouriques, Emanuela G Cartaxo, and Patrícia DL Machado. 2018. Test case prioritization techniques for model-based testing: a replicated study. Software Quality Journal (2018), 1--32.
Patrick Rempel and Parick Mäder. 2017. Preventing defects: The impact of requirements traceability completeness on software quality. IEEE Transactions on Software Engineering 43, 8 (2017), 777--797.
Per Runeson and Björn Regnell. 1998. Derivation of an integrated operational profile and use case model. In Software Reliability Engineering, 1998. Proceedings. The Ninth International Symposium on. IEEE, 70--79.
John Ferguson Smart. 2014. BDD in Action. Manning Publications.
Hany EL Yamany and Miriam AM Capretz. 2007. A Multi-Agent Framework for Building an Automatic Operational Profile. In Advances and Innovations in Systems, Computing Sciences and Software Engineering. Springer, 161--166.
Publicado
23/09/2019
Como Citar
FAZZOLINO, Rafael; RODRIGUES, Genaína Nunes.
Feature-Trace: Generating Operational Profile and Supporting Testing Prioritization from BDD Features. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE (SBES), 33. , 2019, Salvador.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
