RAVEN: Enhancing Test Request Reliability through Real-Time Verification in High-Complexity Systems
Resumo
In embedded software projects for mobile devices, test requests are critical to ensuring product quality, yet complex dependencies on binaries, version control, and country-specific configurations often cause submission errors. These errors lead to inefficiencies and delays in testing pipelines. This paper reports on the development and evaluation of RAVEN (Request Assessment and Verification Engine), a real-time verification system built through an Action Research approach. RAVEN integrates data from JIRA, version control systems, and binary baselines to automatically verify over 30 criteria as users fill out request forms. Deployed in an industrial environment, RAVEN improved process efficiency—reducing approval lead time by 58%, halving submission iterations—and enhanced reliability, lowering the rejection rate of a key request type by 17 percentage points. It also improved the clarity of verification feedback and fostered greater requester accountability. A TAM-based questionnaire confirmed high user-perceived usefulness and ease of use. While some verification gaps remain, RAVEN demonstrated the value of proactive, educational verification. Future work includes extending its scope, integrating ML-based recommendations, and replicating the approach in other contexts.
Palavras-chave:
Automated Testing, Real-Time Verification, Software process efficiency
Referências
2023. Android Open Source Project. [link] Accessed: 2023/04/13.
Abda Albuquerque, Alice Castro, and Heryck Barbosa. 2024. Reports on using an ABB robot in Android mobile testing automation. In Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST). SBC, 74–76.
Ewerton Andrade, Janislley Sousa, Matheus Lopes, Davi Barbosa, Wesllen Lima, and Jimmy Lacerda. 2024. Experiences with Google Approval Process: an automated approach to enhancing efficiency in Android releases. In Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST). SBC, 83–85.
Heryck Michael Dos Santos Barbosa, Pedro Ivo Pereira Lancellotta, Joao Gabriel C Santos, Abda Myrria De Albuquerque, and Janislley O De Sousa. 2023. PRIMA: an Automated Tool for Android Releases Homologation Review. In Congresso Brasileiro de Software: Teoria e Prática (CBSoft). SBC, 1–4.
Nourhan Elsayed, Luka Abb, Heike Sander, and Jana-Rebecca Rehse. 2023. Automating computer software validation in regulated industries with robotic process automation. In International Conference on Business Process Management. Springer, 135–148.
Carol Fernandes, Charles Araujo, Adriano Oliveira, Wellington Correa, Luan Ipê, and Paulo Andrade. 2023. TSS Tool: Automation Tool Applied in the True Single SKU Setup Environment Preparation for Multiple Devices in Parallel. In Proceedings of SAST 2023. [link]
Andy Field. 2024. Discovering statistics using IBM SPSS statistics. Sage publications limited.
E. M. Goldratt and J. Cox. 2004. The Goal: A Process of Ongoing Improvement (3 ed.). North River Press. [link]
Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model context protocol (mcp): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503.23278 (2025).
Bilal Ahmed Kamal, Abdul Basit Shahid, Syed Muhammad Sajjad, Kashif Kifayat, and Khwaja Mansoor Ul Hassan. 2024. A Novel Technique of Android Stealth Rooting. In 2024 17th International Conference on Development in eSystem Engineering (DeSE). IEEE, 275–280.
Aman Kumar, Deepak Narayan Gadde, Keerthan Kopparam Radhakrishna, and Djones Lettnin. 2025. Saarthi: The First AI Formal Verification Engineer. arXiv preprint arXiv:2502.16662 (2025).
Pedro Ivo Pereira Lancellotta, Heryck Michael Dos Santos Barbosa, João Gabriel Castro Santos, Klirssia Matos Isaac Sahdo, and Janislley O De Sousa. 2022. An industry case study: Methodology application to the reviewing process on android releases homologation. In Congresso Brasileiro de Software: Teoria e Prática (CBSoft). SBC, 13–16.
Victoria J Mabin and Robert Y Cavana. 2024. A framework for using Theory of Constraints thinking processes and tools to complement qualitative system dynamics modelling. System Dynamics Review 40, 4 (2024), e1768.
Kai Petersen and Claes Wohlin. 2009. Context in industrial software engineering research. International Symposium on Empirical Software Engineering and Measurement (2009), 401–404.
R. S. Pressman. 2014. Software Engineering: A Practitioner’s Approach. McGraw-Hill. [link]
Partha Pratim Ray. 2025. A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions. Authorea Preprints (2025).
Robert Sanders. 1987. The Pareto principle: its use and abuse. Journal of Services Marketing 1, 2 (1987), 37–40.
Vincenzo Stoico. 2021. A model-driven approach for early verification and validation of embedded systems. In 2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). IEEE, 684–688.
Claes Wohlin, Per Runeson, Martin Höst, Magnus Ohlsson, Björn Regnell, and AndersWesslén. 2012. Experimentation in Software Engineering. Springer Science & Business Media.
Abda Albuquerque, Alice Castro, and Heryck Barbosa. 2024. Reports on using an ABB robot in Android mobile testing automation. In Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST). SBC, 74–76.
Ewerton Andrade, Janislley Sousa, Matheus Lopes, Davi Barbosa, Wesllen Lima, and Jimmy Lacerda. 2024. Experiences with Google Approval Process: an automated approach to enhancing efficiency in Android releases. In Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST). SBC, 83–85.
Heryck Michael Dos Santos Barbosa, Pedro Ivo Pereira Lancellotta, Joao Gabriel C Santos, Abda Myrria De Albuquerque, and Janislley O De Sousa. 2023. PRIMA: an Automated Tool for Android Releases Homologation Review. In Congresso Brasileiro de Software: Teoria e Prática (CBSoft). SBC, 1–4.
Nourhan Elsayed, Luka Abb, Heike Sander, and Jana-Rebecca Rehse. 2023. Automating computer software validation in regulated industries with robotic process automation. In International Conference on Business Process Management. Springer, 135–148.
Carol Fernandes, Charles Araujo, Adriano Oliveira, Wellington Correa, Luan Ipê, and Paulo Andrade. 2023. TSS Tool: Automation Tool Applied in the True Single SKU Setup Environment Preparation for Multiple Devices in Parallel. In Proceedings of SAST 2023. [link]
Andy Field. 2024. Discovering statistics using IBM SPSS statistics. Sage publications limited.
E. M. Goldratt and J. Cox. 2004. The Goal: A Process of Ongoing Improvement (3 ed.). North River Press. [link]
Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model context protocol (mcp): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503.23278 (2025).
Bilal Ahmed Kamal, Abdul Basit Shahid, Syed Muhammad Sajjad, Kashif Kifayat, and Khwaja Mansoor Ul Hassan. 2024. A Novel Technique of Android Stealth Rooting. In 2024 17th International Conference on Development in eSystem Engineering (DeSE). IEEE, 275–280.
Aman Kumar, Deepak Narayan Gadde, Keerthan Kopparam Radhakrishna, and Djones Lettnin. 2025. Saarthi: The First AI Formal Verification Engineer. arXiv preprint arXiv:2502.16662 (2025).
Pedro Ivo Pereira Lancellotta, Heryck Michael Dos Santos Barbosa, João Gabriel Castro Santos, Klirssia Matos Isaac Sahdo, and Janislley O De Sousa. 2022. An industry case study: Methodology application to the reviewing process on android releases homologation. In Congresso Brasileiro de Software: Teoria e Prática (CBSoft). SBC, 13–16.
Victoria J Mabin and Robert Y Cavana. 2024. A framework for using Theory of Constraints thinking processes and tools to complement qualitative system dynamics modelling. System Dynamics Review 40, 4 (2024), e1768.
Kai Petersen and Claes Wohlin. 2009. Context in industrial software engineering research. International Symposium on Empirical Software Engineering and Measurement (2009), 401–404.
R. S. Pressman. 2014. Software Engineering: A Practitioner’s Approach. McGraw-Hill. [link]
Partha Pratim Ray. 2025. A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions. Authorea Preprints (2025).
Robert Sanders. 1987. The Pareto principle: its use and abuse. Journal of Services Marketing 1, 2 (1987), 37–40.
Vincenzo Stoico. 2021. A model-driven approach for early verification and validation of embedded systems. In 2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). IEEE, 684–688.
Claes Wohlin, Per Runeson, Martin Höst, Magnus Ohlsson, Björn Regnell, and AndersWesslén. 2012. Experimentation in Software Engineering. Springer Science & Business Media.
Publicado
22/09/2025
Como Citar
LIMA, Rayfran Rocha; SILVA, Rodrigo Carvalho.
RAVEN: Enhancing Test Request Reliability through Real-Time Verification in High-Complexity Systems. In: SIMPÓSIO BRASILEIRO DE TESTES DE SOFTWARE SISTEMÁTICO E AUTOMATIZADO (SAST), 10. , 2025, Recife/PE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 10-17.
DOI: https://doi.org/10.5753/sast.2025.13677.
