Extraction of test cases procedures from textual use cases

it is worth it?

Authors

  • Erick Barros dos Santos Federal University of Ceará
  • Lucas Simão da Costa Federal University of Ceará
  • Thaís Marinho de Amorim Federal University of Ceará
  • Bruno Sabóia Aragão Federal University of Ceará
  • Ismayle de Sousa Santos Federal University of Ceará
  • Danilo Reis de Vasconcelos Federal Institute of Ceará
  • Rossana Maria de Castro Andrade Federal University of Ceará

DOI:

https://doi.org/10.5753/jserd.2020.731

Keywords:

Software Testing, Test Generation, Test Factory

Abstract

Software testing plays a major role in the quality of software since it assures that the software complies with its expected behavior. However, this is an expensive activity and, thus, companies usually do not perform testing activities on software projects due to the time required. These costs may be even higher in testing processes that rely only on manual test execution, which is both time-consuming and error-prone. One strategy commonly used to mitigate these costs is to use tools to automate testing activities such as test execution, test documentation, and test case generation. This paper presents an experience report in the context of a Test Factory about the use of a tool that partially automates the specification of test procedures from textual use cases. This tool automatically retrieves use cases from the requirement management system, generates the test case procedures, requires inputs from the tester, and then sends the test cases to the test management system. This paper details how this tool was used in real releases of an industrial software project. We also evaluated our tool with four test analysts from a different project to gather more data regarding its efficiency to support the test case documentation.

Downloads

Download data is not yet available.

References

R. M. d. C. Andrade, I. d. S. Santos, V. Lelli, K. M.de Oliveira, and A. R. Rocha. Software testing process ina test factory­from ad hoc activities to an organizational standard. In ICEIS (2), pages 132–143, 2017.

B. Aragão, I. Santos, T. Nogueira, L. Mesquita, and R. An­drade. Modelagem interativa de um processo de desen­volvimento com base na percepção da equipe: Um relatode experiência. InAnais do XIII Simpósio Brasileiro deSistemas de Informação, pages 428–435. SBC, 2017.

B. S. Aragão, R. M. C. Andrade, I. S. Santos, R. N. S. Cas­tro, V. Lelli, and T. G. R. Darin. Testdcat: Catalog of testdebt subtypes and management activities. In Testing Soft­ware and Systems, pages 279–295, Cham, 2019. Springer International Publishing. ISBN 978­3­030­31280­0.

A.Bertolino. Softwaretestingresearch: Achievements, challenges, dreams. In 2007 Future of Software Engineering,FOSE ’07, pages 85–103, Washington, DC, USA, 2007. IEEE Computer Society. ISBN 0­7695­2829­5. URL http://dx.doi.org/10.1109/FOSE.2007.25.

R. M. de Castro Andrade, I. de Sousa Santos, V. Lelli, K. M. de Oliveira, and A. R. C. da Rocha. Software testing pro­cess in a test factory ­ from ad hoc activities to an organi­zational standard. In ICEIS, 2017.

D. Freudenstein, M. Junker, J. Radduenz, S. Eder, and B.Hauptmann. Automatedtest­designfromrequirements­the specmate tool. In 018 IEEE/ACM 5th Interna­tional Workshop on Requirements Engineering and Test­ing (RET), pages 5–8. IEEE, 2018.

V. Garousi and M. Felderer. Living in two different worlds: A comparison of industry and academic focus areas in software testing.IEEE Software, (1):1–1, 2017.

V. Garousi and M. V. Mäntylä. When and what to automatein software testing? a multi­vocal literature review.Infor­mation and Software Technology, 76:92–117, 2016.

T. Gorschek, P. Garre, S. Larsson, and C. Wohlin. A modelfor technology transfer in practice.IEEE software, 23(6):88–95, 2006.

J. Gutiérrez, M. Escalona, and M. Mejías. A model­drivenapproach for functional test case generation.Journal ofSystems and Software, 109:214–228, 2015.

ISO/IEC29119­2. Iso/iec/ieee international standard ­ soft­ware and systems engineering software testing part2:test processes.ISO/IEC/IEEE 29119­2:2013(E), pages1–68, Sept 2013.

D. N. Jorge, P. D. Machado, E. L. Alves, and W. L. Andrade. Integrating requirements specification and model­based testing in agile development. In2018 IEEE 26thInternational Requirements Engineering Conference(RE),pages 336–346. IEEE, 2018.

L. Lazicand N. Mastorakis. Cost effective software test metrics. WSEAS Transactions on Computers, 7(6):599–619,2008.

J. L. Massollar, R. M. de Mello, and G. H. Travassos. Structuring and verifying requirement specifications through activity diagrams to support the semi­automated generation of functional test procedures. In 2012 Eighth International Conference on the Quality of Information and Communications Technology, pages 239–244. IEEE, 2012.

A. Mette and J. Hass. Testing processes. In Software Testing Verification and Validation Workshop, 2008. ICSTW’08.IEEE International Conference on, pages 321–327. IEEE,2008.

M. A. Montoni, A. R. Rocha, and K. C. Weber. Mps. br: a successful program for software process improvement in brazil Software Process: Improvement and Practice, 14(5):289–300, 2009

G. J. Myers, T. Badgett, T. M. Thomas, and C. Sandler.Theart of software testing, volume 2. Wiley Online Library,2004.

S. Nogueira, H. Araujo, R. Araujo, J. Iyoda, and A.Sampaio. Test case generation, selection and coverage from natural language. Science of Computer Programming, pages 84–110, 2019.

J. Preece, Y. Rogers, and H. Sharp. Interaction design. Apogeo Editore, 2004.

G. Samarthyam, M. Muralidharan, and R. K. Anna. Understanding test debt. In Trends in Software Testing, pages1–17. Springer, 2017.

E. B. d. Santos, L. S. d. Costa, B. S. Aragão, I. d. S. Santos,and R. M. d. C. Andrade. Extraction of test cases pro­cedures from textual use cases to reduce test effort: Test factory experience report. In Proceedings of the XVIII Brazilian Symposiumon Software Quality,pages266–275,2019.

K. Schwaber and M. Beedle.Agile software developmentwith Scrum, volume 1. Prentice Hall Upper Saddle River,2002.

S. Seela and R. Yackel. 64 essential testing metrics for mea­suring quality assurance success. URL https://www.qasymphony.com/blog/64-test-metrics/.

H. M. Sneed. Requirement­based testing­extracting logicaltest cases from requirement documents. In International Conference on Software Quality, pages 60–79. Springer,2018.

S. S. Some and X. Cheng. An approach for supporting system­level test scenarios generation from textual usecases. InProceedings of the 2008 ACM symposium onApplied computing, pages 724–729. ACM, 2008.

L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. InAnais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018

L. S. Vieira, C. G. L. Barreto, E. B. dos Santos, B. S. Aragão, I. de Sousa Santos, and R. M. C. Andrade. Automaçãode testes em uma fábrica de testes: Um relato de experiência. In Anais do XIV Simpósio Brasileiro de Sistemas deInformação, pages 80–73. SBC, 2018b.

K. Wiegers and J. Beatty. Software requirements. Pearson Education, 2013.

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in software engineering. Springer Science & Business Media, 2012.

T. Yue, S. Ali, and M. Zhang. Rtcm: a natural language based, automated, and practical test case generation framework. In Proceedings of the 2015 International Sympo­sium on Software Testing and Analysis, pages 397–408.ACM, 2015

Downloads

Published

2020-11-29

Issue

Section

Research Article