The Use of Grey Literature Review as Evidence for Software Engineering

• Fernando K. Kamei UFPE/IFAL
• Sergio Soares UFPE
• Gustavo Pinto UFPA

DOI: https://doi.org/10.5753/cbsoft_estendido.2019.7656

Resumo

Context: In the last years, the Grey Literature (GL) had a notable increase in interest in diverse areas of knowledge. In Software Engineering (SE), with the advance of social media, the SE practitioners are heavy consumers of GL, opposite to traditional research papers, such as the Systematic Literature Review (SLR). Problem: Despite the increase of numbers of SLRs, some researchers claim the lack of connection to the practice of SE, due to the lack of evidence derived from SE practitioners. Goal: Introduce methods and techniques to support a Grey Literature Review (GLR) in SE, aiming to approximate the literature review to the practice of SE. Method: First, we will investigate how secondary studies use GL; Second, we will employ a survey with researchers of SLRs and SE practitioners; Third, we will analyze the use of GL according to the context; Fourth, we will review the GL source in SE; Finally, we will propose a guideline to help SE researchers and practitioners to conduct the GLR. Preliminary findings: We have initiated the tertiary study conducting the manual (4,875 studies) and automatic (6,279 studies) searches. We eliminated those that were not an article, and also those were not written in English, remaining 8,019 potential studies. Currently, we are applying the inclusion and exclusion criteria.

Palavras-chave: Grey Literature, Systematic Literature Review, Grey Literature Review

Referências

Arsenio Paez. Gray literature: An important resource in systematic reviews. Journal of Evidence-Based Medicine, 10(3):233–240, aug 2017.

Richard J. Adams, Palie Smart, and Anne Sigismund Huff. Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4):432–454, apr 2016.

Ashley Williams. Using reasoning markers to select the more rigorous software practitioners online content when searching for grey literature. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018 (EASE ’18). ACM Press, jun 2018.

Fabio Q. B. da Silva, André L. M. Santos, Sérgio Soares, A. César C. França, Cleviton V. F. Monteiro, and Felipe Farias Maciel. Six years of systematic literature reviews in software engineering: An updated tertiary study. Information and Software Technology, 53(9):899–913, sep 2011.

Vahid Garousi, Michael Felderer, and Mika V. Mäntylä. The need for multivocal literature reviews in software engineering. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering (EASE ’16). ACM Press, jun 2016.

Bruno Cartaxo, Gustavo Pinto, Danilo Ribeiro, Fernando Kamei, Ronnie E. S. Santos, Fabio Q. B. da Silva, and Sérgio Soares. Using q&a websites as a method for assessing systematic reviews. In Proceedings of the 14th International Conference on Mining Software Repositories (MSR ’17). IEEE, may 2017.

Margaret-Anne Storey, Alexey Zagalsky, Fernando Figueira Filho, Leif Singer, and Daniel M. German. How social and communication channels shape and challenge a participatory culture in software development. IEEE Transactions on Software Engineering, 43(2):185–204, feb 2017.

Jeffrey C. Carver, Edgar Hassler, Elis Hernandes, and Nicholas A. Kraft. Identifying barriers to the systematic literature review process. In Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’13). IEEE, oct 2013.

Vahid Garousi, Michael Felderer, and Mika V. Mäntylä. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and Software Technology, 106:101–121, feb 2019.

Barbara A. Kitchenham, Tore Dybå, and Magne Jørgensen. Evidence-based software engineering. In Proceedings of 26th International Conference on Software Engineering (ICSE ’04). IEEE Comput. Soc, may 2004.

Gordon Guyatt. Evidence-based medicine. JAMA, 268(17):2420, nov 1992.

Barbara A. Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. Systematic literature reviews in software engineering - a systematic literature review. Information and Software Technology, 51(1):7–15, jan 2009.

Daniela S. Cruzes and Tore Dybå. Research synthesis in software engineering: A tertiary study. Information and Software Technology, 53(5):440–455, may 2011.

Barbara A. Kitchenham, Pearl Brereton, Mark Turner, Mahmood Niazi, Stephen Linkman, Rialette Pretorius, and David Budgen. The impact of limited search procedures for systematic literature reviews - a participant-observer case study. In Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM ’09). IEEE, oct 2009.

Affan Yasin and Muhammad Ijlal Hasnain. On the quality of grey literature and its use in information synthesis during systematic literature reviews. Master’s thesis, Blekinge Institute of Technology, Karlskrona, Sweden, 2012.

Vahid Garousi and Mika V. Mäntylä. When and what to automate in software testing? a multi-vocal literature review. Information and Software Technology, 76:92–117, aug 2016.

Håvard Myrbakken and Ricardo Colomo-Palacios. Devsecops: A multivocal literature review. In Communications in Computer and Information Science, pages 17–29. Springer International Publishing, oct 2017.

Vahid Garousi, Michael Felderer, and Tuna Hacaloğlu. Software test maturity assessment and test process improvement: A multivocal literature review. Information and Software Technology, 85:16–42, may 2017.

Päivi Raulamo-Jurvanen, Mika V. Mäntylä, and Vahid Garousi. Choosing the right test automation tool. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering (EASE ’17). ACM Press, jun 2017.

Barbara A. Kitchenham, Rialette Pretorius, David Budgen, O. Pearl Brereton, Mark Turner, Mahmood Niazi, and Stephen Linkman. Systematic literature reviews in software engineering - a tertiary study. Information and Software Technology, 52(8):792–805, aug 2010.

Barbara A. Kitchenham and Stuart Charters. Guidelines for performing systematic literature reviews in software engineering. techreport, Keele University and Durham University Joint Report, jul 2007.