Improving Traceability Recovery Between Bug Reports and Manual Test Cases

• Lucas Raniére Juvino Santos UFCG
• Guilherme Gadelha UFCG
• Franklin Ramalho UFCG
• Tiago Massoni UFCG

Resumo

Many software tasks produce text. These artifacts are co-dependent, but maintaining their consistency is challenging; automation is desirable. Research has investigated traceability between bug reports and manual test cases. Since manual system test scripts are a popular way of documenting requirements in agile projects, this kind of traceability allows, for instance, to analyze how bugs are related to requirements. Previous work has assessed three Information Retrieval (IR) techniques (LSI, LDA, and BM25) and a Deep-Learning (DL) algorithm (Word Vector) to recover those links; results indicate the need for improvements in textual processing and representation. In this paper, we applied five improvement techniques to an existing data set of bug reports and manual test cases from Mozilla Firefox. We employ: (i) text and information cleaning, (ii) spell-checking, (iii) terms weighting, (iv) similarity matrices merging, and (v) traceability matrices merging. Merging was applied to matrices produced by IR and DL techniques. We evaluate the techniques by comparing precision, recall, and f-measures, with those reached by previous work as a baseline. We observe a slight increase in precision and recall rates for all traceability recovery techniques (LSI, LDA, BM25, and Word Vector) by combining text and information cleaning, title duplication, and spell-checking. A hybrid strategy, creating a merged traceability matrix containing all traces recovered by at least one of the four recovery techniques, achieved a recall value of 93%.

Palavras-chave: bug reports, software artifacts, test cases, traceability