Automated Test Script Generation for Model-Based Testing

• Bernardo Copstein PUCRS
• Flávio Oliveira PUCRS

DOI: https://doi.org/10.5753/sbqs.2005.16167

Resumo

Statistical testing based on a usage model is a key technique for realistic reliability estimation. Since this technique requires large samples (test suites), test execution is automated via scripts and script-based execution engines. However, such scripts are too complex to design manually, both because of the sample size and the complexity of the usage model, which is normally represented with a Markov chain. At the CPTS (Software Testing Research Center), we developed an approach that automates two steps of script design: test suite generation and converting test cases into scripts. The only manual step is the usage model design, which is represented in two models: a stochastic automata network (SAN) for representing the usage model and an intermediate model (called the Interface Event-State Model) to map the abstract usage model into the implemented interface components. SANs allow modular representation of systems with complex non-eterministic behavior, minimizing the state-space explosion found in Markov chains. The use of a separate model for the implementation has the advantage of making changes in the interface and generating the script automatically without affecting the abstract usage model. We implemented the technique into the STAGE environment; the experiments indicate that the modeling task takes less than one day for a typical model with 420 global states and 8 days for one with over 200.000 global states, while the test suite and script generation takes less than 1 minute. Also, the performance of test case and script generation with SANs is at least compatible with Markov-based generation.

Palavras-chave: Automated Test, Script Generation, Model-Based Testing

Referências

Farina, A.G.; Fernandes, P H L; Oliveira, F.M.. “Representing Usage Models With Stochastic Automats Networks”. In: 14th International Conference On Software Engineering And Knowledge Engineering - SEKE02, 2002, Ischia, Italy.

Fernandes, P. Méthodes numériques pour la solution de systèmes Markoviens à grand espace d’ états. INPG, Grenoble, 1998. (PhD Thesis)

Fernandes, P., Plateau, B. and Stewart, W.J. Efficient descriptor-vector multiplication in stochastic automata networks. Journal of the ACM, volume 45, no. 3, 1998, pp. 281-414.

J.M.Fourneau, B.Plateau. PEPS: a package for solving complex Markov models of parallel systems. In: Proceedings of the Fourth International Conference on Modelling Techniques and Tools for Computer Performance Evaluation. Palma, Spain, 1988.

Fujiwara, S.; Bochmann G. V.; Khendek F.; Amalou M. and Ghedemsi A. Test Selection Based on Finite State Model. IEEE Transactions On Software Engineering, vol 17(6):591-603, 1991.

Kallepalli, C.; Tian J. - Measuring and Modeling Usage and Reliability for Statistical Web Testing. IEEE Transactions on Sotware Engineering, vol 27(11):1023- 1036, 2001.

Rational products. Description available at URL http://www.rational.com/products/. Accessed 9/19/2003.

Sayre, K. Improved Techniques for Software Testing Based on Markov Chain Usage Models. PhD thesis. University of Tennessee, Knoxville, December 1999.

Shehady, R.K.; Siewiorek, D.P.; A method to automate user interface testing using variable finite state machines Fault-Tolerant Computing, 1997. FTCS-27. Digest of Papers., Twenty-Seventh Annual International Symposium on Fault-Tolerant Computing, 24-27 June 1997. Page(s): 80 -88

Trammel, C. Quantifying the Reliability of Software: Statistical Testing Based on a Usage Model. Proceedings of the Second IEEE International Symposium on Software Engineering Standards, Canada, August 1995.

Beyer, M; Dulz, W; Zhen, F. Automated TTCN-3 Test Case Generation by means of UML Sequence Diagrams and Markov Chains. Proceedings of the 12th Asian Test Symposium (ATS ’03), 2003.