A Protocol Based on Systematic Mapping Studies for Searching Health Applications in Non-Academic Repositories





Systematic Mapping Study, Health Applications, Application Repositories, Source Code Repositories


Systematic mapping studies (SMSs) help obtain an overview of a research topic. SMSs are conducted with protocols that ensure their reliability and allow their replication by other researchers. The focus of an SMS is to identify peer-reviewed articles indexed in academic databases. Therefore, conducting an SMS does not encompass identifying software applications hosted in non-academic repositories. This article presents a protocol for systematic mappings in non-academic repositories (SMNARs). The SMNAR protocol was developed from adaptations of an SMS methodology consolidated in the literature and was aimed at assisting in the search, cataloging, and analysis of health software applications hosted in non-academic application and source code repositories. Thus, part of the adaptations was based on health application search approaches found in the literature. Two SMNAR user guides were developed, one designed for healthcare professionals and the other for software developers. To assess the feasibility of these guides, a study was conducted with participants from the healthcare and computer science fields. This study had both synchronous and asynchronous moments. In the former, a presentation contextualized the SMNAR protocol, proposed guides, and study procedures. In the latter, the participants partially planned and conducted an SMNAR using the proposed guides as support. Online forms were applied to obtain participants' perceived ease of use, perceived usefulness, and intention to use the proposed guides. The participants indicated good acceptance of the guides and suggested possible improvements in the protocol description. These results contributed to refining the SMNAR user guides.


Download data is not yet available.


Arksey, H. and O'Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1):19-32. DOI: 10.1080/1364557032000119616.

Azevedo, L. F. and Sousa-Pinto, B. (2019). Avaliação crítica de uma revisão sistemática e meta-análise: Da definição da questão de investigação à pesquisa de estudos primários. Revista da Sociedade Portuguesa de Anestesiologia, 28(1):53-56. DOI: 10.25751/rspa.17320.

Basili, V. R. and Rombach, H. D. (1988). The tame project: Towards improvement-oriented software environments. IEEE Transactions on Software Engineering, 14(6):758-773. DOI: 10.1109/32.6156.

Bonato, S. (2022). Grey literature searching. Piecing Together Systematic Reviews and Other Evidence Syntheses, page 111. ISBN: 1538150174.

Brasil, L. M. (2008). Informática em Saúde. EDUEL. ISBN: 8572164944.

Brown, H. M., Bucher, T., Collins, C. E., and Rollo, M. E. (2020). A review of pregnancy apps freely available in the google play store. Health Promotion Journal of Australia. DOI: 10.1002/hpja.270.

Corbin, J. and Strauss, A. (2014). Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications. ISBN: 1412997461.

Costa, P. H., Cordeiro, A. F., and OliveiraJr, E. (2021). Comparing open data repositories. In Anais da V Escola Regional de Engenharia de Software, pages 60-69. SBC. DOI: 10.5753/eres.2021.18451.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, pages 319-340. DOI: 10.2307/249008.

Dermeval, D., Coelho, J. A. P. d. M., and Bittencourt, I. I. (2019). Mapeamento sistemático e revisão sistemática da literatura em informática na educação. Metodologia de Pesquisa em Informática na Educação: Abordagem Quantitativa de Pesquisa. Porto Alegre: SBC. ISBN: 978-85-7669-494-6.

Field, A. P. and Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63(3):665-694. DOI: 10.1348/000711010X502733.

Formagini, T. D. B., Ervilha, R. R., Machado, N. M., Andrade, B. A. B. B. d., Gomide, H. P., and Ronzani, T. M. (2017). Revisão dos aplicativos de smartphones para cessação do tabagismo disponíveis em língua portuguesa. Cadernos de Saúde Pública. DOI: 10.1590/0102-311X00178215.

Garg, S. and Baliyan, N. (2021). Comparative analysis of Android and iOS from security viewpoint. Computer Science Review, 40:100372. DOI: 10.1016/j.cosrev.2021.100372.

Garousi, V., Felderer, M., and Mäntylä, M. V. (2019). Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and Software Technology, 106:101-121. DOI: 10.1016/j.infsof.2018.09.006.

Henson, P., David, G., Albright, K., and Torous, J. (2019). Deriving a practical framework for the evaluation of health apps. The Lancet Digital Health, 1(2):e52-e54. DOI: 10.1016/S2589-7500(19)30013-5.

Kamei, F., Wiese, I., Pinto, G., Ribeiro, M., and Soares, S. (2020). On the use of grey literature: A survey with the brazilian software engineering research community. In Proceedings of the 34th Brazilian Symposium on Software Engineering, pages 183-192. DOI: 10.1145/3422392.3422442.

Kiatake, L. G. G., Junior, L. A. V., da Silva, M. L., and Sanzovo, O. A. C. (2020). Manual de certificação de sistemas de registro eletrônico em saúde. Sociedade Brasileira de Informática em Saúde, pages 1-33. Technical report, Version 5.0.

Kitchenham, B. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical report, Version 2.3, EBSE-2007-01.

Laurent, A. M. S. (2004). Understanding open source and free software licensing: guide to navigating licensing issues in existing & new software. O'Reilly Media, Inc. ISBN: 9780596005818.

Lima, L. F., Huve, C. A. G., and Peres, L. M. (2020). Software product quality evaluation guide for electronic health record systems. In Proceedings of the 34th Brazilian Symposium on Software Engineering, pages 108-113. DOI: 10.1145/3422392.3422478.

Mafra, S. N. and Travassos, G. H. (2006). Estudos primários e secundários apoiando a busca por evidência em engenharia de software. Technical report, RT-ES 687/06.

Mahood, Q., Van Eerd, D., and Irvin, E. (2014). Searching for grey literature for systematic reviews: challenges and benefits. Research synthesis methods, 5(3):221-234. DOI: 10.1002/jrsm.1106.

Marcelo, C. A. d. S., Coutinho, M. A. P., Lara, C. R., Paraizo, C. M. S., and Fava, S. M. C. L. (2020). Aplicativos m'oveis sobre diabetes mellitus - revisão narrativa. Journal of Health Informatics, 12(2). ISSN: 2175-4411.

Morais, E. R., Vergara, C. M. A. C., Brito, F. O. d., and Sampaio, H. A. d. C. (2020). Serious games para educação em higiene bucal infantil: uma revisão integrativa e a busca de aplicativos. Ciência & Saúde Coletiva, 25:3299-3310. DOI: 10.1590/1413-81232020258.11782018.

Petersen, K., Vakkalanka, S., and Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. DOI: 10.1016/j.infsof.2015.03.007.

Querino, J. d. J., Andrade, N. N., Santos, G. B. d., and Santos, L. C. (2020). Levantamento e avaliação de aplicativos sobre ansiedade e depressão em língua portuguesa. Revista de Saúde Digital e Tecnologias Educacionais, 5(2):72-88. DOI: 10.36517/resdite.v5.n2.2020.a6.

Reynolds, C. J. and Wyatt, J. C. (2011). Open source, open standards, and health care information systems. Journal of Medical Internet Research, 13(1). DOI: 10.2196/jmir.1521.

Rosenthal, R. and DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology, 52(1):59-82. DOI: 10.1146/annurev.psych.52.1.59.

Scotini, C. Y., Orsati, F. T., and Macedo, E. C. d. (2021). Levantamento e avaliação da acessibilidade de aplicativos para apoio ao aprendizado de crianças com diagn'ostico de transtorno do espectro autista. Cadernos de P'os-Graduação em Distúrbios do Desenvolvimento, 21(1):39-67. DOI: 10.5935/cadernosdisturbios.v21n1p39-67.

Silveira, L. K., de Carvalho, L. O., Rosa, L. F. F., Paraizo, C. M. S., Dázio, E. M. R., and Fava, S. M. C. L. (2020). Aplicativos m'oveis sobre hipertensão arterial sistêmica: revisão narrativa. Brazilian Journal of Development, 6(2):7413-7422. DOI: 10.34117/bjdv6n2-153.

Souza, C. M. d. and Silva, A. N. (2016). Aplicativos para smartphones e sua colaboração na capacidade funcional de idosos. Revista de Saúde Digital e Tecnologias Educacionais, 1(1):6-19. ISSN: 2525-9563.

Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., and Mani, M. (2015). Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth, 3(1):e27. DOI: 10.2196/mhealth.3422.

Trecca, E. M. C., Lonigro, A., Gelardi, M., Kim, B., and Cassano, M. (2021). Mobile applications in otolaryngology: A systematic review of the literature, Apple App Store and the Google Play Store. Annals of Otology, Rhinology & Laryngology, 130(1):78-91. DOI: 10.1177/0003489420940350.

Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D., Horsley, T., Weeks, L., et al. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7):467-473. DOI: 10.7326/M18-0850.

Venkatesh, V. and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2):186-204. DOI: 10.1287/mnsc.

Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, pages 1-10. DOI: 10.1145/2601248.2601268.

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media. DOI: 10.1007/978-3-642-29044-2.

Ye, X., Zheng, Y., Aljedaani, W., and Mkaouer, M. W. (2021). Recommending pull request reviewers based on code changes. Soft Computing, 25(7):5619-5632. DOI: 10.1007/s00500-020-05559-3.




How to Cite

Lima, L. F. de, & Peres, L. M. (2022). A Protocol Based on Systematic Mapping Studies for Searching Health Applications in Non-Academic Repositories. SBC Reviews on Computer Science, 2(1). https://doi.org/10.5753/reviews.2022.2724