Proposal and Evaluation of a Collaborative IS to Support Systematic Reviews and Mapping Studies
Resumo
Context: Systematic Literature Review (SLR) or Systematic Mapping Study (SMS) are a process in which publications dataset is systematically analyzed to cover a research field. These processes involve multiple investigators collaborating to produce more improved work and often use automated tools to facilitate their work. Problem: However, not all tools offer proper support to collaborative SLR or SMS. That is, missing a tool to support the study selection process, allowing the collaboration between researchers by applying individual criteria and collective decision, supported by agreement or discussion and consensus. Solution: We developed the Porifera tool to fill this lack. IS Theory: Technology Acceptance Model (TAM) and a Grounded Theory's phase subset were used to evaluate Porifera's tool quality. Methodology: Undergraduate and postgraduate students enrolled in the Experimental Software Engineering Research,used the Porifera tool and answered a post-used questionnaire with TAM's sentences and other open questions. Then, a quantitative and qualitative analysis was performed. Summary of Results: It was possible to see high perceived usefulness and ease of use for Porifera. Too it noted the effectiveness of resources to support the collaborative activity and its contribution to learning and performing a collaborative SLR or SMS. The evaluation showed points to improve the Porifera's interface. Contribution and Impact in the IS area: The Porifera is an IS for data, information, and knowledge research management because it gathers publications records and allows it will be interpreted and processed, making possible decisions making by researchers. The Porifera also allows performing an SLR or SMS with mobility, knowledge sharing, flexibility, and integration between people and technology.
Palavras-chave:
Systematic Literature Review, Systematic Mapping Study, SLR tool, collaborative system, user feedback, software evaluation
Referências
Ahmed Al-Zubidy and Jeffrey C. Carver. 2019. Identification and prioritization of SLR search tool requirements: an SLR and a survey. Empirical Software Engineering 24, 1 (feb 2019), 139–169. https://doi.org/10.1007/s10664-018-9626-5
Victor R. Basili and Gianluigi Caldiera. 2000. The Goal Question Metric Paradigm. Encyclopedia of Software Engineering - 2 Volume Set 2 (2000), 528–532. https://www.cs.umd.edu/~basili/publications/technical/T89.pdf
Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4thed.). Sage Publishing, San Jose, USA. 456 pages. https://us.sagepub.com/en-us/nam/basics-of-qualitative-research/book235578
Fred D. Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies 38, 3 (mar 1993), 475–487. https://doi.org/10.1006/IMMS.1993.1022
Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (nov 1971), 378–382. https://doi.org/10.1037/H0031619
Edgar Hassler, Jeffrey C. Carver, David Hale, and Ahmed Al-Zubidy. 2016. Identification of SLR tool needs - Results of a community workshop. Information and Software Technology 70 (feb 2016), 122–129. https://doi.org/10.1016/j.infsof.2015.10.011
Elis Hernandes, Augusto Zamboni, Sandra Fabbri, and André Di Thommazo. 2012. Using GQM and TAM to evaluate StArt – a tool that supports Systematic Review. CLEI Electronic Journal 15, 1 (2012), 3–3. https://doi.org/10.19153/cleiej.15.1.2
Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical Report. Keele University and University of Durham. 65 pages. https://doi.org/10.1145/1134285.1134500 arxiv:1304.1186
Barbara Kitchenham, Rialette Pretorius, David Budgen, O. Pearl Brereton, Mark Turner, Mahmood Niazi, and Stephen Linkman. 2010. Systematic literature reviews in software engineering-A tertiary study., 792–805 pages. https://doi.org/10.1016/j.infsof.2010.03.006
Barbara Ann Kitchenham, David Budgen, and Pearl Brereton. 2015. Evidence-Based Software Engineering and Systematic Reviews (1st ed.). Chapman and CRC Press, Boca Raton, USA. 433 pages. https://doi.org/10.1201/b19467
Christian Kohl, Emma J. McIntosh, Stefan Unger, Neal R. Haddaway, Steffen Kecke, Joachim Schiemann, and Ralf Wilhelm. 2018. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: A case study on CADIMA and review of existing tools. Environmental Evidence 7, 1 (2018), 1–17. https://doi.org/10.1186/s13750-018-0115-5
Luciano Marchezan, Guilherme Bolfe, Elder Rodrigues, Maicon Bernardino, and Fábio Paulo Basso. 2019. Thoth: A Web-based Tool to Support Systematic Reviews. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Vol. 1. Institute of Electrical and Electronics Engineers (IEEE), Porto de Galinhas, Brazil, 3–8. https://doi.org/10.1109/ESEM.2019.8870160
Christopher Marshall. 2016. Tool support for systematic reviews in software engineering. Ph. D. Dissertation. Keele University, Keele, Newcastle, UK.
Christopher Marshall and Pearl Brereton. 2015. Systematic review toolbox: A catalogue of tools to support systematic reviews. In EASE ’15: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Vol. 1. Association for Computing Machinery, Nanjing, China, 1–6. https://doi.org/10.1145/2745802.2745824
Christopher Marshall, Pearl Brereton, and Barbara Kitchenham. 2014. Tools to support systematic reviews in software engineering: A feature analysis. In EASE ’14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. Association for Computing Machinery, London, UK, 1–10. https://doi.org/10.1145/2601248.2601270
Jefferson Seide Molléri and Fabiane Barreto Vavassori Benitti. 2015. SESRA - A web-based automated tool to support the systematic literature review process. In EASE ’15: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Vol. 1. Association for Computing Machinery, Nanjing, China, 1–6. https://doi.org/10.1145/2745802.2745825
Jakob Nielsen. 2000. Why You Only Need to Test with 5 Users., 4 pages. https://doi.org/alertbox/20000319.html
Chitu Okoli and Kira Schabram. 2010. A Guide to Conducting a Systematic Literature Review of Information Systems Research. SSRN Electronic Journal(2010). https://doi.org/10.2139/ssrn.1954824
Mourad Ouzzani, Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid. 2016. Rayyan-a web and mobile app for systematic reviews. Systematic Reviews 5, 1 (2016), 1–10. https://doi.org/10.1186/s13643-016-0384-4
Parsifal Ltd.2018. About · Parsifal. https://parsif.al/about/
Kai Petersen and Nauman Bin Ali. 2011. Identifying strategies for study selection in systematic reviews and maps. International Symposium on Empirical Software Engineering and Measurement (2011), 351–354. https://doi.org/10.1109/ESEM.2011.46
Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic mapping studies in software engineering. In EASE’08: Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering. BCS Learning and Development Ltd., Bari, Italy, 1–10. https://doi.org/10.14236/ewic/ease2008.8
Mark Petticrew and Helen Roberts. 2005. Systematic Reviews in the Social Sciences: A Practical Guide (1st ed.). Wiley-Blackwell. 354 pages. [link].
Rebeca de Souza e Silva and Ângela Tavares Paes. 2012. Por Dentro da Estatística: teste de concordância de Kappa. Educ. Contin. Saúde einstein 10, 4 (2012), 165–166.
Victor R. Basili and Gianluigi Caldiera. 2000. The Goal Question Metric Paradigm. Encyclopedia of Software Engineering - 2 Volume Set 2 (2000), 528–532. https://www.cs.umd.edu/~basili/publications/technical/T89.pdf
Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4thed.). Sage Publishing, San Jose, USA. 456 pages. https://us.sagepub.com/en-us/nam/basics-of-qualitative-research/book235578
Fred D. Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies 38, 3 (mar 1993), 475–487. https://doi.org/10.1006/IMMS.1993.1022
Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (nov 1971), 378–382. https://doi.org/10.1037/H0031619
Edgar Hassler, Jeffrey C. Carver, David Hale, and Ahmed Al-Zubidy. 2016. Identification of SLR tool needs - Results of a community workshop. Information and Software Technology 70 (feb 2016), 122–129. https://doi.org/10.1016/j.infsof.2015.10.011
Elis Hernandes, Augusto Zamboni, Sandra Fabbri, and André Di Thommazo. 2012. Using GQM and TAM to evaluate StArt – a tool that supports Systematic Review. CLEI Electronic Journal 15, 1 (2012), 3–3. https://doi.org/10.19153/cleiej.15.1.2
Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical Report. Keele University and University of Durham. 65 pages. https://doi.org/10.1145/1134285.1134500 arxiv:1304.1186
Barbara Kitchenham, Rialette Pretorius, David Budgen, O. Pearl Brereton, Mark Turner, Mahmood Niazi, and Stephen Linkman. 2010. Systematic literature reviews in software engineering-A tertiary study., 792–805 pages. https://doi.org/10.1016/j.infsof.2010.03.006
Barbara Ann Kitchenham, David Budgen, and Pearl Brereton. 2015. Evidence-Based Software Engineering and Systematic Reviews (1st ed.). Chapman and CRC Press, Boca Raton, USA. 433 pages. https://doi.org/10.1201/b19467
Christian Kohl, Emma J. McIntosh, Stefan Unger, Neal R. Haddaway, Steffen Kecke, Joachim Schiemann, and Ralf Wilhelm. 2018. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: A case study on CADIMA and review of existing tools. Environmental Evidence 7, 1 (2018), 1–17. https://doi.org/10.1186/s13750-018-0115-5
Luciano Marchezan, Guilherme Bolfe, Elder Rodrigues, Maicon Bernardino, and Fábio Paulo Basso. 2019. Thoth: A Web-based Tool to Support Systematic Reviews. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Vol. 1. Institute of Electrical and Electronics Engineers (IEEE), Porto de Galinhas, Brazil, 3–8. https://doi.org/10.1109/ESEM.2019.8870160
Christopher Marshall. 2016. Tool support for systematic reviews in software engineering. Ph. D. Dissertation. Keele University, Keele, Newcastle, UK.
Christopher Marshall and Pearl Brereton. 2015. Systematic review toolbox: A catalogue of tools to support systematic reviews. In EASE ’15: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Vol. 1. Association for Computing Machinery, Nanjing, China, 1–6. https://doi.org/10.1145/2745802.2745824
Christopher Marshall, Pearl Brereton, and Barbara Kitchenham. 2014. Tools to support systematic reviews in software engineering: A feature analysis. In EASE ’14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. Association for Computing Machinery, London, UK, 1–10. https://doi.org/10.1145/2601248.2601270
Jefferson Seide Molléri and Fabiane Barreto Vavassori Benitti. 2015. SESRA - A web-based automated tool to support the systematic literature review process. In EASE ’15: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, Vol. 1. Association for Computing Machinery, Nanjing, China, 1–6. https://doi.org/10.1145/2745802.2745825
Jakob Nielsen. 2000. Why You Only Need to Test with 5 Users., 4 pages. https://doi.org/alertbox/20000319.html
Chitu Okoli and Kira Schabram. 2010. A Guide to Conducting a Systematic Literature Review of Information Systems Research. SSRN Electronic Journal(2010). https://doi.org/10.2139/ssrn.1954824
Mourad Ouzzani, Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid. 2016. Rayyan-a web and mobile app for systematic reviews. Systematic Reviews 5, 1 (2016), 1–10. https://doi.org/10.1186/s13643-016-0384-4
Parsifal Ltd.2018. About · Parsifal. https://parsif.al/about/
Kai Petersen and Nauman Bin Ali. 2011. Identifying strategies for study selection in systematic reviews and maps. International Symposium on Empirical Software Engineering and Measurement (2011), 351–354. https://doi.org/10.1109/ESEM.2011.46
Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic mapping studies in software engineering. In EASE’08: Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering. BCS Learning and Development Ltd., Bari, Italy, 1–10. https://doi.org/10.14236/ewic/ease2008.8
Mark Petticrew and Helen Roberts. 2005. Systematic Reviews in the Social Sciences: A Practical Guide (1st ed.). Wiley-Blackwell. 354 pages. [link].
Rebeca de Souza e Silva and Ângela Tavares Paes. 2012. Por Dentro da Estatística: teste de concordância de Kappa. Educ. Contin. Saúde einstein 10, 4 (2012), 165–166.
Publicado
16/05/2022
Como Citar
CAMPOS, Thiago Prado de; DAMASCENO, Eduardo Filgueiras; VALENTIM, Natasha Malveira Costa.
Proposal and Evaluation of a Collaborative IS to Support Systematic Reviews and Mapping Studies. In: SIMPÓSIO BRASILEIRO DE SISTEMAS DE INFORMAÇÃO (SBSI), 18. , 2022, Curitiba.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
