Exploratory testing of apps with opportunity maps

• Rubens Copche UTFPR
• Mariana Souza Slingshot Digital
• Isabel Karina Villanes UFAM
• Vinicius Durelli UFSJ
• Marcelo Eler USP
• Arilo Cláudio Dias-Neto UFAM
• Andre Takeshi Endo UTFPR

Resumo

Exploratory Testing (ET) is a well-known technique to perform manual tests. Its main goal is to foster creativity and freedom, while the tester learns, designs and executes tests continuously in a time-boxed session. Popular among agile teams, ET is particularly interesting for software systems with highly-mutable features like mobile apps. Previous results have evinced that testers may fail to verify well-known features of mobile apps, while performing unguided manual testing. This paper investigates an intervention to exploratory tests in which opportunity maps are adopted as a mean to improve the manual testing of mobile apps. Included as a supporting artifact during the test sessions, opportunity maps are mind maps with questions targeting features of apps that are known to be error-prone. To assess the usage of opportunity maps in ET, we conducted a study with 22 participants and compared the proposed approach with a traditional session-based approach. Our results indicated that the number of detected bugs was similar in both approaches, different bugs were revealed by each approach, and opportunity maps tend to guide the detection of specific bugs. Among the participants, we found that practitioners uncovered more bugs than students.

Palavras-chave: Mobile Applications, Manual Tests, Android, Session-based Testing