Proceedings of the Brazilian Software Quality Symposium (SBQS)
Sociedade Brasileira de Computação - SBC
Will AI also replace inspectors? Investigating the potential of generative AIs in usability inspection
Usability inspection is a well-established technique for identifying interaction issues in software interfaces, thereby contributing to improved product quality. However, it is a costly process that requires time and specialized knowledge from inspectors. With advances in Artificial Intelligence (AI), new opportunities have emerged to support this task, particularly through generative models capable of interpreting interfaces and performing inspections more efficiently. This study examines the performance of generative AIs in identifying usability problems, comparing them to those of experienced human inspectors. A software prototype was evaluated by four specialists and two AI models (GPT-4o and Gemini 2.5 Flash), using metrics such as precision, recall, and F1-score. While inspectors achieved the highest levels of precision and overall coverage, the AIs demonstrated high individual performance and discovered many novel defects, but with a higher rate of false positives and redundant reports. The combination of AIs and human inspectors produced the best results, revealing their complementarity. These findings suggest that AI, in its current stage, cannot replace human inspectors but can serve as a valuable augmentation tool to improve efficiency and expand defect coverage. The results provide evidence based on quantitative analysis to inform the discussion on the role of AI in usability inspections, pointing to viable paths for its complementary use in software quality assessment contexts.
Software Developers’ Perceptions of Productivity: An Industry-focused Study
Context: Developer productivity is a critical factor of software project success and, by extension, a key driver of organizational performance and software quality in software engineering. In the era of Generative Artificial Intelligence (GenAI), understanding the developers’ perception of productivity becomes paramount for a software organization seeking to adopt GenAI tools. Objective: This paper aims to examine developers’ perception of productivity, focusing on its measurement and the expected impact of adopting GenAI. Method: We conducted two semi-structured focus groups with software developers from a large science and technology institute. We also administered a confirmatory questionnaire to support the verification of our findings. The collected data were analyzed using content analysis. Results:We identified five key productivity factors: quality, expectations, deadlines, progress, and efficiency, including distinct measurement systems for coding, code review, and documentation activities. Participants consistently emphasized the potential of GenAI to streamline and enhance a range of routine development tasks, albeit with certain limitations regarding their use. Conclusion: Our findings indicate that productivity constitutes a complex multidimensional construct influenced by interrelated factors. Moreover, GenAI emerge as a transformative technology whose impacts on productivity require deeper investigation in future studies, particularly regarding balancing efficiency gains with the preservation of technical abilities.
Return of the QA: A Comparative Analysis of Competencies and Skills for Quality Analysts in Brazil (2021–2025)
Background: The software industry has undergone rapid changes driven by agile methodologies, continuous delivery pipelines, and increased automation. As system complexity and quality demands grow, the role of Quality Analysts (QAs) has become increasingly critical. This shift is particularly relevant in the current context, characterized by the rapid evolution of QA practices, technological advancements, and the increasing adoption of AI-based tools that provide assisted support throughout the software development process. Objectives: This study aims to identify the key technical and behavioral competencies that characterize high-performing QAs, from the perspective of experienced practitioners in the Brazilian software industry. Method:We carried out two large-scale surveys involving Brazilian companies, conducted in 2021 and 2025, with a total of 169 participants. Results: The findings reveal a strong and growing demand for soft skills, especially communication and critical thinking, alongside solid technical foundations. Respondents emphasized the evolving and multidisciplinary nature of the QA role in modern software development contexts. Conclusions: This research offers a comprehensive and recent overview of industry expectations regarding QA analyst competencies in Brazil. The results provide actionable insights for those seeking to advance in the QA field, educators aiming to align curricula with market demands, and organizations aiming to strengthen their software quality assurance capabilities.
Prioritization of ISO/IEC 25012 Data Quality Dimensions Using the AHP Technique
This article presents a method for prioritizing data quality dimensions based on the ISO/IEC 25012 standard, using the Analytic Hierarchy Process (AHP) technique. The method assists in assessing data quality for training predictive models, considering the relevance of dimensions according to the application context. The research combines qualitative analysis, through the collection of expert judgments, and quantitative analysis to calculate relative weights between the selected dimensions. To assess the method’s reproducibility and applicability, a simulation was conducted for a predictive model in hospital bed management. The simulation results indicate the feasibility of guiding data quality improvement actions and strengthening information governance for use in predictive modeling.
May I speak? Perceptions on ethical concerns and power while developing software in AI teams
Context: As Artificial Intelligence (AI) technologies increasingly influence decision-making processes, ethical concerns in AI development have gained significant attention. However, most research in this area focuses on high-level decision-makers or large technology corporations in the Global North, overlooking the lived experiences of software engineers in underrepresented contexts. Goal: This study investigates how Brazilian software engineers working in AI teams at a mid-sized company perceive ethical principles, navigate ethical dilemmas, and respond to emerging ethical concerns in their everyday work. The focus is on understanding how Junior, Mid-Level, and Senior developers interpret and apply ethical frameworks during the concrete development of AI systems such as classifiers, image generators, and object detectors. Method: A mixed-methods approach was adopted, combining data from 18 survey responses and 8 in-depth semi-structured interviews. The qualitative data were analyzed using a combination of inductive and deductive coding to identify recurring patterns, ethical challenges, and coping strategies. Results: Findings indicate that while participants are generally aware of key ethical principles—such as fairness, transparency, and accountability—their ability to act on ethical concerns is limited by factors including organizational hierarchies, lack of formal ethical training, and insufficient autonomy in design decisions. Ethical deliberation is often informal, reactive, and constrained by resource and power asymmetries. Conclusion: This research highlights the ethical agency of AI practitioners in the Global South and contributes to expanding the geographical and professional scope of AI ethics literature. It emphasizes the urgent need for organizational structures that empower developers to raise, discuss, and resolve ethical issues throughout the AI development lifecycle.
Impact of Generative Artificial Intelligence on Knowledge Management in Software Engineering: A Systematic Mapping Study
Context: In recent years, Generative Artificial Intelligence (GenAI) has emerged as a transformative technology with high potential across various organizational contexts. Leveraging techniques such as Natural Language Processing (NLP) and Large Language Models (LLMs), tools like ChatGPT, GitHub Copilot, and DALL-E have begun to facilitate information creation and retrieval, automate tasks, and optimize decision-making processes. In the field of Knowledge Management (KM), GenAI has served as an enabler for organizing, disseminating, and reusing knowledge, fostering more collaborative and responsive environments. Specifically in Software Engineering, GenAI demonstrates potential to enhance productivity. The integration of GenAI with KM practices can play a key role in improving software quality, by enabling better reuse of knowledge, supporting consistent practices, and promoting informed decision-making. Understanding how GenAI can support KM in this domain is therefore essential to guide its strategic and effective integration. Objective: This study aims to investigate the impact of GenAI on KM processes within organizations operating in the Software Engineering domain. Method: To achieve this, we conducted a systematic mapping study to identify current practices, perceptions, and challenges related to the use of GenAI in KM within Software Engineering. Results: The study analyzed 97 primary studies published between 2021 and 2025. Main results indicate that GenAI focusing on code generation and software construction activities. KM practices most commonly supported include knowledge application and creation, with externalization emerging as the dominant SECI knowledge conversion mode. Conclusion: GenAI contributes significantly to KM in Software Engineering by supporting operational tasks and the formalization of knowledge. However, limited attention to knowledge dissemination and deeper learning processes reveals promising directions for future research.
How Far From the Metaphor? The Diverse Understandings of Technical Debt in GitHub Issues
This study investigates how the metaphor of technical debt (TD) is used in practice to communicate complex aspects of software development, particularly in relation to quality-delivery trade-offs. The study addresses the problem of inadequate or inconsistent application of the TD concept, sometimes compounded by linguistic, cultural, and organizational factors that hinder effective communication and compromise project sustainability. A qualitative analysis of issues from GitHub repositories was conducted using the Qualitative Data Analysis method. The findings revealed four main categories of TD interpretation: Decision, Activity, Problem and Pending Task. These categories encompass eight usage contexts. Many of the observed interpretations diverge significantly from the original metaphor, reinforcing the need for clearer conceptual alignment to support technical debt management practices.
Heuristics for Integrated Interaction (HII): A New Heuristic Evaluation Technique to Identify Usability Problems in HInt Devices
Background: The complexity of Human-Computer Integration (HInt) devices poses challenges for traditional usability assessment methods, as they involve elements that go beyond the interface, demanding new forms of evaluation. These technologies work in symbiosis with the human body and operate autonomously with the user in real time, requiring specific methods that can identify potential failures. Aims: This article introduces the Heuristics for Integrated Interaction (HII) technique, designed to identify usability problems in HInt technologies by using criteria tailored to the integration paradigm. Method: The technique was developed based on a literature review, followed by an empirical study with experts. The experts applied HII to assess a real-world system, reporting their perceptions about its applicability and effectiveness. Results: The results indicated that HII allowed the identification of specific failures not captured by traditional approaches. Participants considered HII’s relevance for integrative contexts and suggested structural and linguistic improvements. Conclusions: The results reinforce the prediction of the technique as an innovative instrument for usability evaluation in HInt devices, thereby contributing to the improvement of the software quality of these systems.
Exploring ChatGPT Efficiency in Automatic Test Generation for Python: A Comparative Analysis
Context: Large language models (LLMs) like ChatGPT have gained attention in automated software testing. This study evaluates ChatGPT-3.5-turbo’s ability to generate test sets for Python programs, comparing it with Pynguin and pre-existing test sets. Problem: Automated testing remains challenging for dynamically typed languages like Python, requiring adaptable tools for diverse code structures. Solution: We assessed ChatGPT-3.5-turbo’s test generation using different prompt configurations and temperature settings. Method: Using 40 Python programs, we generated Pytestcompliant tests via the OpenAI API, varying temperature settings (0.0 to 1.0). Tests were validated using Pytest, with coverage and mutation scores measured via Coverage, MutPy, and Cosmic-Ray. Pynguin-generated and pre-existing test sets served as baselines. Summary of Results: ChatGPT-3.5-turbo successfully generated valid tests for simpler programs, but averaged below 28% overall, with a low cost. Higher temperatures (0.5–1.0) improved results, but combining test cases from all temperatures introduces diversity in the LLM-generated test sets, making it possible to overcome both Pynguin and pre-existing test sets in terms of decision coverage and mutation score.
Exploratory Evaluation of Secure Design Methodologies Between Security and Code Quality
This paper investigates the impacts of secure design methodologies on the structural quality and presence of vulnerabilities in systems developed in Java. We analyzed 333 popular GitHub repositories, classified according to four security approaches: Security by Design, Defense in Depth, Runtime Security and Zero Trust Architecture. Using static analysis techniques, we evaluated vulnerabilities based on the CWE taxonomy, in addition to code quality metrics such as coupling (CBO), cohesion (LCOM) and cyclomatic complexity. The results indicate that the adoption of secure design practices, although relevant for risk mitigation, can negatively impact the modularity of the system. In particular, approaches such as Security by Design presented higher levels of coupling and lower cohesion. We also observed that the combination of multiple security practices did not necessarily result in a lower number of vulnerabilities, suggesting an increase in complexity without proportional benefit. These findings highlight the importance of balanced architectural decisions that consider the commitments between security and internal software quality.
Evaluating the Effectiveness and Cost-Efficiency of Large Language Models in Automated Unit Test Generation
Software quality plays a crucial role in the development process, and one of the main strategies to ensure it is through software testing. However, testing can be a costly and time-consuming task, leading researchers to explore ways to automate it. This study analyzes the ability of large language models (LLMs) to generate unit tests automatically, evaluating their performance based on four key criteria: line coverage, branch coverage, hit rate, and cost. To achieve this, unit tests were generated using three different input formats: (i) source code only, (ii) source code with a docstring, and (iii) a detailed prompt with step-by-step instructions. The results show that all models produce tests with high line and branch coverage. However, hit rates vary depending on the input format. The lowest hit rate was 72.61%, obtained by ChatGPT-4o Mini when provided with only the source code. The highest hit rate was 90.69%, achieved by Gemini 1.5 Pro when given source code with a docstring, a performance close to the 96.81% hit rate of the dataset’s reference tests. The findings indicate that more affordable models, such as Gemini 1.5 Flash and GPT-4o Mini, can achieve competitive results when optimized prompts are used, making them cost-effective alternatives for unit test generation.
Evaluating LLM-Generated Unit Tests with Mutation Testing: ChatGPT vs DeepSeek
Recent advances in Large Language Models (LLMs) have driven significant progress in automating software testing, particularly in generating unit tests. However, the effectiveness of these models in detecting real defects through mutation testing remains underexplored in the literature. This study aims to address this gap by evaluating the performance of ChatGPT (GPT-4o) and DeepSeek V3 in generating unit tests for six Java classes from the Defects4J dataset, covering different levels of cyclomatic complexity. The main objective is to investigate the ability of LLMs to maximize mutant coverage and elimination, while also analyzing the impact of code complexity and semantic factors related to execution failures. The methodology involved generating tests via structured prompts, executing them 5 times per class for both models, and performing quantitative analysis based on Mutation Coverage (MC) and Mutation Score (MS), as well as qualitative analysis of runtime failures. Results indicate that DeepSeek exhibits greater stability and effectiveness in eliminating mutants, whereas ChatGPT demonstrates broader applicability by producing valid test suites for a wider range of classes. Moreover, no significant correlation was found between cyclomatic complexity and compilation success, with failures primarily linked to semantic limitations of the models. This study presents both quantitative and qualitative evidence on the application of LLMs for automated test generation, offering insights for future AI-driven test engineering strategies.
DIP-AI: A Discovery Framework for AI Innovation Projects
Despite the increasing development of Artificial Intelligence (AI) systems, Requirements Engineering (RE) activities face challenges in this new data-intensive paradigm.We identified a lack of support for problem discovery within AI innovation projects. To address this, we propose and evaluate DIP-AI, a discovery framework tailored to guide early-stage exploration in such initiatives. Based on a literature review, our solution proposal combines elements of ISO 12207, 5338, and Design Thinking to support the discovery of AI innovation projects, aiming at promoting higher quality deliveries and stakeholder satisfaction. We evaluated DIP-AI in an industry-academia collaboration (IAC) case study of an AI innovation project, in which participants applied DIP-AI to the discovery phase in practice and provided their perceptions about the approach’s problem discovery capability, acceptance, and suggestions. The results indicate that DIP-AI is relevant and useful, particularly in facilitating problem discovery in AI projects. This research contributes to academia by sharing DIP-AI as a framework for AI problem discovery. For industry, we discuss the use of this framework in a real IAC program that develops AI innovation projects.
Designed for a Court, Used in a University: The Communicability of SEI from the Perspective of Undergraduate Students
This work analyzes the communicability of the Electronic Information System (SEI) from the perspective of undergraduate students at a Brazilian public university. Originally developed for the Judiciary, SEI has been adopted by several public institutions, including universities, where its use has been extended to processes involving students. To assess the system’s communicative quality, a survey was conducted with 37 students, who identified SEI as the most complex digital tool among those analyzed.We applied the Semiotic Inspection Method (SIM) to a representative use case, revealing inconsistencies in the interface’s metacommunication. Based on these findings, interface improvements were proposed and subsequently evaluated by information technology specialists. Results indicate that the revised version enhances the system’s communicability and usability, providing a more intuitive user experience. The work underscores the importance of adapting institutional systems to real usage contexts, especially in academic environments.
Systematic Mapping of Data Quality in the Public Sector: Dimensions, Metrics and Practices
Data quality plays a critical role in ensuring the reliability and effectiveness of information systems in public sector organizations. However, fragmented data ecosystems, legacy systems, and a lack of standardized assessment practices challenge the implementation of consistent data quality strategies. This study presents a comprehensive systematic mapping of data quality dimensions, metrics, practices, and challenges across public institutions. A total of 53 peer-reviewed studies were analyzed, revealing the most frequently used data quality dimensions and associated metrics. The research highlights a wide spectrum of techniques, ranging from traditional validation rules to emerging applications of machine learning for anomaly detection and data imputation. In addition, it identifies key barriers — including limited automation, lack of governance, and scarce resources — and proposes actionable recommendations for public sector entities. The findings serve as a conceptual and practical foundation for improving data quality management in complex government environments, with the ultimate goal of supporting the implementation of data quality assessment strategies in a state-level public finance institution in Brazil, the Ceará State Treasury Department (Sefaz-CE).
Client-Side vs Server-Side Rendering: Impacts on Performance and User Experience in Web Applications
The increasing complexity of web applications necessitates careful selection of rendering strategies to ensure optimal performance and user experience. This study presents a comparative analysis of Client-Side Rendering (CSR) and Server-Side Rendering (SSR) techniques, evaluating their impact on web application quality based on Google’s Core Web Vitals metrics (CWV). In this work, photo gallery web applications were implemented using both rendering approaches to allow a homogeneous environment for comparable results. Through controlled experiments across five data sizes (ranging from 498 kB to 26 MB), measurements were taken for key performance indicators, including First Contentful Paint (FCP), Speed Index (SI), Time to Interactive (TTI), Largest Contentful Paint (LCP), and JavaScript Heap Used. Statistical analysis using the Kruskal-Wallis test revealed significant performance differences (𝑝 ≤ 0.05) in all metrics. Results show that SSR achieves superior performance in TTI (mean reduction of 62.3%) and LCP (mean reduction of 58.7%), particularly under increased data loads, while CSR exhibits better FCP and SI performance for smaller datasets. The results suggest that the choice between CSR and SSR should consider the application’s specific objectives and usage environment. These findings contribute to evidence-based decision-making frameworks for web application architecture, supporting software quality engineering practices in modern development environments.
Evaluation and verification of requirements artifacts: A Checklist applied to User Stories and Prototype
The quality of system requirements is essential to ensure that the software meets user needs. In the context of software engineering, specifying requirements through user stories and prototypes has become a widely adopted practice in the industry, as it facilitates effective communication among requirements, development, and testing teams. However, for these practices to be effective, information essential to developing functionality must be clearly specified and organized. To support this verification process and ensure the completeness of artifacts, this work proposes VUSP (Checklist for the Assessment and Verification of User Stories and Prototypes), focused on the verification of requirements artifacts. The feasibility of VUSP was evaluated in a study involving 38 participants from a Software Engineering course at the Federal University of Amazonas, who used the checklist to verify user stories and prototypes. Participants provided feedback on the tool’s usability and effectiveness using the TAM (Technology Acceptance Model), addressing aspects such as perceived ease of use, work relevance, and behavioral intention. The results indicate that VUSP helped identify improvements in documentation and the centralization of essential information for development.
Assessing Quality Through Use: A Usability-Based Evaluation of the Conta gov.br Authentication Platform
This article presents an empirical study that evaluated the Conta gov.br authentication platform, a nationally adopted service in Brazil, focusing on usability as a dimension of software quality. Thirty participants took part in usability testing using standardized metrics: System Usability Scale (SUS), Single Ease Question (SEQ), and Task Completion Time (TCT). Results showed variations in perceived usability influenced by demographic factors such as age and education level. Statistical correlations between scores and user profiles were also analyzed. Based on the findings, a Practical Guide was developed to support public institutions in their adoption strategies. The results contribute evidence-based insights to inform quality-oriented decisions in digital public services.
ARTeMIS: Agent-based Rewriting and Test Case Management with Intelligent Supervision
The quality of Test Case (TC) scripts is essential for the execution and automation of test scenarios on mobile devices. It is common to find TC components that are out of date, poorly written or inconsistent with documentation standards. In this work, we propose ARTeMIS, a modular Multi-Agent framework designed to automate the rewriting and validation of non-standardized TC components, integrating semantic retrieval, supervised classification, structured prompting and iterative rule-based validation using Large Language Models (LLMs). Our framework is capable of standardizing these three TC components: Summary, Initial Setup and Test Steps. ARTeMIS assigns each component to specialized agents for classification, rewriting and validation, enabling syntactic consistency and semantic accuracy with minimal human intervention. We performed experiments using, in addition to ARTeMIS, three other LLM-based techniques well established in the literature: Zero-Shot, Few-Shot and Retrieval-Augmented Generation (RAG). Our experiments demonstrated the feasibility and extensibility of our approach, which achieved a higher accuracy compared to the other techniques, highlighting the potential of agent-based architectures to standardize and automate TCs in continuous industrial testing environments.
An Ontology for Safety Assurance Information Management
Context: Diverse information needs to be traced and managed throughout the development lifecycle of safety-critical systems, such as requirements, design definitions, tests and code. A safety-critical system (SCS) undergoes certification procedures that require the presentation of arguments supporting the design and implementation decisions involved in the SCS construction. Nowadays, many regulations require elaborating assurance cases (ACs) to present such arguments. The integrated management of the diverse data required to produce the ACs can leverage the continuous traceability assessment and support the automated generation of ACs at any time in the SCS lifecycle. Objective: Ontologies can integrate metamodels to connect data from different domains. We designed and implemented a Safety Assurance Ontology (SAO) to integrate outputs of safety and software engineering activities. SAO encompasses an information model, rules and competency questions to support the work of software, safety and assurance engineers in the context of an SCS development that requires the production of ACs. Method: SAO was built based on knowledge extracted from safety datasets and argumentation patterns. This work presents the SAO’s design, implementation, and evaluation through an illustrative scenario with public datasets and an expert survey with industrial practitioners and researchers. Results: SAO comprises concepts, relationships, and formation rules integrating safety assurance and software project information. We implemented the ontology information model and its formation rules using semantic web technologies. Conclusions: SAO can trace relevant project information, infer implicit information, and highlight traceability gaps in the public datasets evaluated. Practitioners find SAO’s competency questions on traceability gaps useful in revealing direct and indirect relationships among project items. They also find that the basic competency questions facilitate the contextualisation of items traced in the project repositories. Also, they pointed out that adding specialisations for key concepts can enhance practitioners’ grasp of the metamodel scope, thus promoting its applicability.
An Approach to Assist the Design of User-Centered Solutions based on Generative AI
User-Centered Design (UCD) aims to ensure software quality by aligning its development with user needs, enabling users to guide project decisions and contribute to the overall quality of the product. Some methodologies, such as Design Thinking and Lean UX, incorporate UCD, but they often lack a rigorous structure that focuses on the rapid validation of solutions. This paper investigates the Design Sprint, a UCD approach, which offers short development cycles for validating solutions with the users. We developed an approach based on generative Artificial Intelligence (AI) to automate Design Sprint activities through structured prompts designed with prompt engineering techniques. We evaluated the proposed approach through a feasibility study involving 38 participants. The study included introductory training and the practical application of the prompts in team-defined projects. Results indicate a predominantly positive reception of the proposed approach, with 85,7% of the teams rating all artifact-generating activities as either “very helpful” or “helpful” during the Design Sprint phases of understanding, sketching, decision-making, and prototyping, which reinforces the feasibility of the approach presented in this study.
AgileMood: A Tool for Ongoing Measurement of Psychological Safety in Agile Teams
Psychological safety and individual emotions have proven essential for the effective functioning of agile teams, directly impacting collaboration, collective performance and ultimately, software quality. This paper presents AgileMood, a tool designed to diagnose and enhance psychological safety and emotional awareness in agile teams through the collection of periodic anonymous feedback, the calculation of specific metrics, and the generation of dynamic visualizations. The solution not only measures the overall perceived level of safety and emotions but also detects internal variations among team members, enabling data-driven interventions. Agile-Mood distinguishes itself from existing approaches by incorporating well-established theoretical foundations, preserving anonymity, and offering practical support for decision-making by team leaders. The empirical evaluation of the tool, conducted through a case study involving 41 students (7 teams of 5 to 8 team members), utilized the System Usability Scale (SUS) and the Technology Acceptance Model (TAM), demonstrating promising usability and high perceived usefulness. Available as open-source software, the tool facilitates replicability and further development by the community. As a contribution, this work proposes a practice-oriented artifact with the potential to transform psychological safety and emotions into a measurable and manageable element within agile software development teams, thereby fostering improved software quality. Future studies should empirically evaluate its application in real organizational contexts.
Advances in Requirements Engineering for Well-Being, Aging, and Health: A Systematic Mapping Study
Context. The increasing focus on well-being, aging, and health (WBAH) is impacting various fields, including Requirements Engineering (RE). This is particularly relevant, as RE involves defining, documenting, and managing software system requirements to ensure that developed systems effectively address user needs. However, there remains a limited understanding of how RE practices can be tailored to create WBAH-centric systems. Aims. This study aims to explore the current landscape of RE practices used in the development of software systems that support WBAH. Method. We conducted a systematic mapping study, analyzing articles from reputable conferences and journals. Results. Our review identified 57 articles published between 2013 and 2024 that examine RE efforts related to specifying and documenting WBAH systems. The predominant topics discussed include usability and evaluation techniques, elicitation and modeling methods, security and privacy concerns for healthcare applications, and the adaptation of RE for specific populations, such as the elderly. Conclusion. This study highlights significant knowledge gaps that present opportunities for further investigation by RE researchers.
Adoption of LLMs in Requirements Engineering: What Practitioners Are Worried About?
Requirements engineers face persistent challenges related to stakeholder communication, information ambiguity, and the precise definition of system needs. In this context, user stories have gained traction as a structured, user-centered approach to representing requirements. At the same time, there is growing interest in using Large Language Models (LLMs) to automate such tasks. However, before proposing automated solutions, it is essential to understand how professionals currently elicit requirements, what techniques and tools they use, and how open they are to adopting LLMs for user story generation. To investigate this, an online survey was conducted with 21 software industry professionals experienced in requirements engineering. The questionnaire, divided into three sections, explored current practices in requirements elicitation and validation, as well as the use of LLMs in these activities. The results showed that 57.1% of participants currently use user stories in their work. Among them, 66.7% reported using LLMs, primarily Chat-GPT, for tasks such as brainstorming and content review. Of these LLM users, 75% also apply them to user story generation, citing benefits such as increased speed, standardization, and improved story quality. However, limitations were also identified, including lack of context, potential over-reliance, and legal concerns. These findings indicate that there is growing interest in adopting LLMs in requirements engineering, including user story generation, as long as their use aligns with established practices and is supported by effective prompting, contextual input, and data privacy safeguards.
A Transparency-oriented BPMN Model Inspection Checklist: From conception to expert evaluations
Process modeling is a key component for the success of an information system. Ensuring the quality of these models is essential for developing effective systems, including accessibility, auditability, understandability, informativeness, and usability. This article presents the design and evaluation of the Transparency-oriented Process Checklist (TPC), a structured inspection instrument for BPMN models emphasizing transparency-related quality characteristics. The TPC was developed by analyzing and integrating items from two existing checklists (CMP and BPCheck), refining them based on transparency dimensions, and structuring it with 51 verification items. A comparative evaluation was conducted to assess the applicability and effectiveness of the TPC. The evaluation team was divided into two groups. One group used the TPC, while the other applied BPCheck to inspect a real-world BPMN model. The results demonstrated that the TPC facilitated the identification of more quality problems (21 vs. 17), achieved greater agreement with the predefined “Oracle” answers (68.63% vs. 36.36%), and received more favorable assessments regarding understandability and support for identifying auditability issues. These findings underscore the TPC’s potential as a valuable instrument for supporting the inspection for the enhancement of BPMN process models.
Impact of MVC and MVVM Architectures on Test Quality: An Analysis of GitHub Open-Source Repositories
This study compared the quality of test code in projects based on the Model-View-Controller (MVC) and Model-View-ViewModel (MVVM) architectures in GitHub repositories. The effectiveness of test suites was evaluated through mutation testing.The research problem lies in the lack of empirical evidence on whether architectural differences, which affect modularity, separation of concerns, and ease of test creation, also translate into higher effectiveness of test suites. To address this, the objective was to evaluate and compare the fault detection capability of tests in both architectures using mutation testing. The methodology involved computing mutation scores, examining the relationship between code coverage and test effectiveness, and analyzing the incidence of mutants that fail due to compile-time errors. The experiments were conducted with Stryker.NET for the automated execution of mutation tests and subsequently applied statistical analyses to interpret the collected data. The study showed no evidence of the impact of architectural choices on test effectiveness. Results also showed a mild correlation between code coverage of tests and mutation score. This finding is relevant, as it indicates that although more modular architectures facilitate the creation of tests, this characteristic does not necessarily translate into higher quality of the test suites.
Lessons Learned from the Use of Generative AI in Engineering and Quality Assurance of a WEB System for Healthcare
The advances and availability of technologies involving Generative Artificial Intelligence (AI) are evolving clearly and explicitly, driving immediate changes in various work activities. Software Engineering (SE) is no exception and stands to benefit from these new technologies, enhancing productivity and quality in its software development processes. However, although the use of Generative AI in SE practices is still in its early stages, considering the lack of conclusive results from ongoing research and the limited technological maturity, we have chosen to incorporate these technologies in the development of a web-based software system to be used in clinical trials by a thoracic diseases research group at our university. For this reason, we decided to share this experience report documenting our development team’s learning journey in using Generative AI during the software development process. Project management, requirements specification, design, development, and quality assurance activities form the scope of observation. Although we do not yet have definitive technological evidence to evolve our development process significantly, the results obtained and the suggestions shared here represent valuable insights for software organizations seeking to innovate their development practices to achieve software quality with generative AI.
Applying Scrumban in Hybrid Software and Hardware Teams: An Experience Report
Context:Agile software development methodologies, such as Scrum, Kanban, and Scrumban, have gained prominence in Software Engineering by promoting greater flexibility and workflow visibility. Scrumban, in particular, combines elements of Scrum and Kanban and represents a promising alternative in contexts that demand continuous adaptation, such as hybrid projects integrating software and hardware development. Goal: This study investigates the perception of a multidisciplinary team regarding the application of Scrumban in hybrid projects involving simultaneous software and hardware development. Method: We conducted a survey with 15 team members from a Technological Development Institute to gather perceptions about using Scrumban in a hybrid context. We qualitatively analyzed the responses to identify reported benefits and challenges. Results: The findings suggest that Scrumban significantly enhances the visibility of task progress and fosters alignment among technical teams through visual boards and communication ceremonies. However, participants reported challenges in applying the methodology to hardware development, such as difficulties in task estimation, supplier dependencies, and the rigidity of physical development cycles. They emphasized the need to adapt agile practices to address the specificities of the hybrid context. Conclusion: The experience demonstrated that Scrumban is a valuable approach to promote integration and transparency in hybrid projects, although it requires adjustments to accommodate the particularities of hardware development. This report reinforces the importance of flexible and adaptive methodologies in multidisciplinary environments.
From Complaints to Requirements: Using NLP and QFD to Derive Use Cases for University Academic Management System
This study introduces a hybrid methodology combining Natural Language Processing (NLP) and Quality Function Deployment (QFD) to enhance the quality and responsiveness of institutional academic systems. The approach systematically transforms unstructured user complaints into functional software requirements, thereby improving documentation and maintenance in legacy systems. By leveraging NLP to extract and prioritize user needs based on dissatisfaction severity and frequency, and translating these into technical requirements via QFD, the method creates a scalable, usercentered, and traceable requirements engineering process, ideal for legacy systems with limited documentation. Analyzing 4,357 service tickets, our results identify critical improvement areas, including authentication, enrollment, and bureaucratic workflows. Although steps like technical mapping and modeling remain manual, the methodology boosts prioritization, transparency, and alignment with essential software quality attributes. While applied to this specific context, the framework shows potential for transferability to other institutional or commercial sectors. These findings demonstrate the value of integrating AI with software engineering principles to modernize complex institutional platforms and increase user satisfaction.
An Approach to Automating User Story Management for Testing Process Optimization in Large Distributed Teams
Efficiently managing user stories during the testing process is crucial in large, distributed test teams to ensure timely and high-quality project completion. Manual control of tasks, from various software testing activities to extracting specific quality metrics and maintaining test processes, is time-consuming and error-prone. This paper proposes an approach to automate story creation for the Jira tool to address these challenges. The proposal approach aims to optimize the time spent on story creation, promote efficiency, and increase the success rate of story creation from 73.43% to 99.66% in the story management process. The script developed establishes a secure connection with Jira, imports data from spreadsheets, populates these into Jira, and creates or updates test tasks. Additionally, JavaScript (JS) code automates data transfer from task creation to planning. This automation significantly reduces manual effort and minimizes human errors, due to the improvement in the success rate to 98.14%, and increases the productivity of the software testing team by 15.19%.
Sidagro Development: An Experience Report on the Partnership between the Federal University of Lavras and the Agricultural Institute of Minas Gerais
This article presents an experience report on the development of Sidagro Web, a governmental system developed through a partnership between the Federal University of Lavras (UFLA) and the Agricultural Institute of Minas Gerais (IMA), with a primary focus on software quality assurance. The report details key architectural and technical decisions that guided the project throughout its lifecycle, providing the rationale for each choice and emphasizing their contributions to enhancing system quality. The discussion is organized into five main categories: (i) high-level architectural choices, including layered separation and foundational technologies; (ii) backend-specific decisions, such as exception handling, object mapping, auditing mechanisms, and code conventions; (iii) frontend considerations, including form validation and dynamic data handling; (iv) infrastructure-related strategies, covering environment setup, CI/CD pipelines, and DevOps tooling; and (v) documentation and version control practices, which are applied transversally across all project dimensions. We demonstrate that the systematic adoption of these practices significantly improved the system’s robustness, maintainability, and long-term sustainability, offering valuable insights for similar large-scale software projects with stringent quality requirements.
Hand-Speaking Technology: An Expert and User-Based Assessment of the VLibrasWidget
The use of assistive technology to support the inclusion of deaf individuals in the digital environment is a valuable resource when it becomes necessary to scale the translation services typically provided by human interpreters. VLibrasWidget is an open-source computational solution designed to translate digital content (text, audio, and video) into Brazilian Sign Language (Libras) through animation. This tool is available on more than 120,000 websites, with an average of 100,000 translations per day. Given the widespread use and social impact of VLibrasWidget, it is important to consider that usability and accessibility aspects are essential for its effective use. Therefore, this experience report presents the planning, execution, and results of an evaluation of VLibrasWidget, combining a heuristic usability review by experts with user testing involving both deaf and hearing participants—through usage observation and subjective satisfaction probing. The complementary results enabled the identification of 28 interface and interaction issues with varying severity levels (1 high; 8 medium; 10 low; 9 cosmetic), indicating that the tool has room to evolve in order to better serve its users.
Automatic Generation of Bug Reports Using Large Language Models: An Evaluation in a Software Institute
Context: During software development, the test team is responsible for verifying if the requirements were correctly implemented. In case the software does not behave as expected, the tester must report the bug to the development team. For the problem to be assertively and quickly, it is important that the bug report is complete, containing all the information needed to fix the bug. This study was conducted in the context of a test team that acts in a software institute, and is responsible for testing mobile devices. Problem: Among their activities, the test team must write bug reports. However, given a high amount of test requests, consequently there will be many bug reports, which will take more time and effort. Goal: With the goal of automating the bug report process, we developed a system that automatically generates bug reports based on the Text-to-Text Transfer Transformer (T5) Large Language Model (LLM). To generate a bug report, the tester must input a prompt containing a brief description of the bug. Method: We also performed an experiment with 8 members from the test team that write bug reports daily, so as to measure their perceptions regarding the solution we developed. The participants used the system and evaluated its outputs for the generation of 5 distinct bug reports. Results: Results showed a high acceptance rate for using the system within the test team, with 3 of the 5 bug types included in the experiment having 87.5% of outputs considered valid or partially valid. Furthermore, participants highlighted the ease of use, the efficacy and quality of writing in the bug reports generated by the system. However, they also noted that the system still needs to be adjusted to work for more types of issues, thus reducing the need for manual fixes. Conclusions: We conclude, then, that it is possible to use LLMs to automatically generate bug reports, however it is still indispensable to have a human review the report after it is generated.
Quality Management in the Public Sector: Definition and Prioritization of Indicators for Assessment and Decision-Making
The use of historical data in software projects, when aligned with quality goals and business objectives, has increasingly been recognized as a strategic approach to enhancing both management and system development. This practice relies on the analysis of collected data, which enables the identification of opportunities and the anticipation of risks. As quality processes are implemented, these data are transformed into measures and metrics used to develop indicators, allowing for continuous performance monitoring and driving organizational improvement. In this context, this report presents the process used for the definition, validation, and prioritization of quality measures and indicators applied to the management and development of software projects in a public organization. Furthermore, in this study we discuss the main challenges faced, as well as the lessons learned and the benefits observed at the current stage of the process of implementing this practice within the institution.
Analysis of Contributions at a Software Institute through the Introduction of a Pre-Trained Model for Requirements Classification
Context: Ensuring that requirements are adequately covered in test cases represents a challenge in the software industry. Specifically, a Software Institute maintains a testing team that continuously analyzes the requirements to ensure their implementation in test cases. Problem: However, requirements analysis is a human-dependent process and faces a large volume of requirements received by the testing team. In addition, other activities compete with requirements analysis, requiring effort and allocation to ensure that a requirement is analyzed and incorporated into the test case. Goal: In order to automate the requirements analysis process, we developed a tool based on XLNet, a pre-trained model for classifying and determining if the requirement is part of the scope of the testing team. Method: To evaluate this tool, we conducted a study with a team of 4 members who analyze requirements, in which the participants conducted the requirements analysis manually and with the aid of the tool. The study consisted of two analyses: (1) quantitative, aimed at evaluating effectiveness (correctly classified requirements) and efficiency (requirements analysis time), and (2) qualitative, in which we developed a questionnaire to obtain feedback from the participants on the use of the tool. Results: In quantitative terms, the statistical tests indicated that there was no significant difference in terms of efficiency between manual and automated classification, with a p-value of 0.8824. Regarding effectiveness, a p-value of 0.0177 was obtained, however, the results showed that manual sorting is still more effective than tool-assisted sorting. Despite this, the qualitative results showed that 100% of the participants agreed that using the tool could improve their performance in the requirements analysis activity and identified positive points about its use, such as accuracy, speed of analysis, and a reduction in the effort dedicated to this activity. Conclusions: The results show that using the tool can bring benefits by automating the analysis and classification of requirements.
Structuring Privacy and Information Security Competencies for Public Sector Roles: A Framework for Enhancing Software Quality and LGPD Compliance
Context: The increasing complexity of digital public services has heightened the need for robust governance mechanisms in privacy and information security, particularly in light of Brazil’s General Data Protection Law (LGPD). However, public institutions often lack clearly defined roles, responsibilities, and competencies for professionals managing sensitive data and system security. Objective: This study aims to develop a comprehensive competency framework that maps the responsibilities and required knowledge, skills, and abilities (KSAs) for key privacy and information security roles in the Brazilian federal public administration. Method: We employed a design science approach, grounded in national regulations (LGPD, PPSI, IN GSI/PR nº 3/2021) and international standards (e.g., ISO/IEC 27701), to analyze legal and normative documents. The process included the identification of 24 institutional roles (8 in privacy, 16 in information security), the modeling of KSAs across three proficiency levels, and the use of the Analytic Hierarchy Process (AHP) to prioritize competencies. Results: The resulting framework provides structured competency profiles for each role, supporting training journey design, maturity assessment, and decision-making for role allocation. An interactive online platform makes the full model publicly accessible, offering practical tools for public sector adoption. Key findings highlight overlapping areas between privacy and security domains, reinforcing the need for coordinated institutional efforts. Conclusion: By clarifying role expectations and aligning them with legal and technical requirements, the framework supports public organizations in improving their institutional maturity in privacy and security governance. It also contributes to the quality, reliability, and trustworthiness of digital public services through strategic capacity-building.
Automated Test Case Generation in a Real-World System Using a Customized AI Agent: An Experience Report
Test case design is an essential activity in software quality assurance. However, when performed manually, it can be time-consuming, error-prone, and require substantial effort, particularly in complex applications. This experience report describes the development and application of an artificial intelligence agent, built using the ChatGPT platform, designed to automate the process of generating test cases for a real-world system and reduce the time required. The agent was configured to simulate the role of a QA analyst, using functional requirements, interface prototypes, and prompt engineering strategies to produce test scenarios with high coverage and accuracy. Experimentswere conducted on one of the modules of a component assembly control system, comparing manually created test cases with those generated by the agent. The results showed a reduction of over 50% in specification time while maintaining the quality and coverage of the scenarios. This paper details the agent’s configuration, the results achieved, the challenges encountered, and the lessons learned, contributing evidence for the practical use of generative AI in the context of software quality assurance.
DevMetrics: An Experience Report on Effort Estimation in Software Maintenance Using a Jira-Integrated Framework
This paper presents the development of DevMetrics, a generic system designed to support effort estimation in software maintenance activities. The tool combines textual similarity (Jaccard index) with statistical regression (linear and exponential), normalized using the Fibonacci sequence, and integrates seamlessly with Jira. A case study was conducted in a medium-sized software house using MMRE and PRED(0.25) metrics, along with Likert-based surveys to assess team perception. Results show that the tool enhances estimation accuracy and planning predictability, especially when teams adopt standardized task descriptions and ensure historical data quality.
Exploring User Experience Factors for Mobile Devices in Textual Data from Questionnaires
The evaluation of User Experience (UX) prior to product launch provides insights into end-user acceptance indicators and helps identify potential issues. Therefore, it is important to conduct studies that evaluate user group perceptions, supported by various methodological approaches. Among these, questionnaires may include questions addressing both quantitative and qualitative aspects of user perception. Textual data related to qualitative aspects can be systematically analyzed to identify UX characteristics that contribute to a broader understanding of user perception. This paper presents an experience report on the classification of UX factors based on textual data collected through questionnaires. The purpose of this practice is driven by the belief that a structured analysis of UX factors can deepen the understanding of such data. We analyzed questionnaire results from two mobile device projects involving a total of 92 users, who were the respondents of the questionnaires. For this classification, 26 practitioners analyzed the textual data reported in the questionnaires. The results indicate that UX factors can provide deeper insights into usability and user satisfaction across different components of the devices. Additionally, the practitioners considered this type of classification useful for understanding the overall quality of the projects through questionnaires.
A Structured Checklist Approach to Evaluating Transparency and Privacy in Brazilian Digital Services
Context: The digitalization of public services in Brazil has improved access and efficiency, but has also raised concerns about transparency, privacy, and data protection. Despite the regulatory framework established by the Brazilian General Data Protection Law (LGPD), many services still lack clear communication and support for citizens regarding how their data is handled. Objective: This study aims to evaluate the trustworthiness of digital public services in Brazil, from the perspective of an ordinary citizen, by analyzing the alignment between declared privacy practices and actual data handling behaviors, as well as the accessibility of support provided to citizens in exercising their data protection rights. Method: We developed and applied a structured evaluation checklist based on three dimensions: (i) Declared Information Quality (QID), (ii) Factual Precision (PF), and (iii) Accessibility Support (AS). Five widely used federal services were selected and analyzed through manual inspection of public documentation, service behavior, and availability of support channels. Each item was scored based on predefined criteria, and qualitative observations were recorded to complement the analysis. Results: The overall average score across all services was 44.76%, with the best performance in factual precision (77.14%) and the worst in privacy support (20.00%). Only one service—“Conta Gov”—achieved a score above 50%. Transparency deficits were common, especially in the disclosure of sensitive data handling and the availability of dedicated support channels or Data Protection Officers (DPOs). Conclusion: The study reveals that while several services functionally comply with privacy expectations, most fail to communicate this effectively to users. The absence of accessible and explicit privacy declarations, combined with limited support mechanisms, undermines citizens’ ability to exercise their rights and reduces trust in digital government services. Our findings highlight the need for standardized privacy documentation and specialized support channels, reinforcing the importance of transparency as a critical component of software quality in the public sector.
Evaluating the Acceptance of Project Management Platforms in Distributed Software Teams: An Experience Report
Geographically distributed software development teams often face challenges in adopting project management tools, with decisions frequently based on comparisons that overlook human and contextual factors. This study investigates the acceptance of GitHub Projects, Huly, and ClickUp within a Brazilian software development organization with distributed collaborators. Specialists from the Software Engineering team participated, representing various organizational roles and product lines. Data were collected through a structured questionnaire, based on indicators from the Technology Acceptance Model (TAM3), and semi-structured interviews. Quantitative and qualitative analyses revealed that ClickUp had the highest acceptance rates, particularly regarding perceived ease of use and perceived usefulness. The results also highlighted that information centralization, platform stability, and feature completeness are critical factors for tool adoption. This research contributes to the field of Software Quality by offering empirical evidence to support decision-making during tool transitions.
Assessing the Use of a Code Generation Assistant in Professional Software Development: An Experience Report
AI-assisted code generation tools, such as GitHub Copilot, have gained attention from the software industry due to their potential to support development tasks. This paper presents an experience report from a case study on the adoption of GitHub Copilot by a team of developers at the Superintendence of Information Technology of the Federal University of Piauí (STI/UFPI), aiming to evaluate the effects of using Copilot in a real-world environment involving legacy system maintenance. The study was carried out over three months, comparing periods with and without the use of the tool. Key metrics such as code change rate and developers’ perceptions were analyzed. The results indicate a increase in the code change rate for most developers, particularly in bug-fixing tasks, while customization tasks showed a more nuanced outcome. Participants also reported positive feedback regarding Copilot’s usefulness. This paper discusses the challenges faced during the study’s execution, methodological limitations, and proposes directions for future research on the impact of AI assistants in diverse software development contexts.
Computing Students’ Perceptions of Ethical Principles in AI and Curricular Coverage
Context: As artificial intelligence (AI) systems become increasingly integrated into society, ensuring that computing professionals are equipped with ethical competencies is a growing concern. While many international guidelines emphasize ethical principles in AI development, their integration into computing curricula remains inconsistent, especially in the Brazilian context. Goal: This study investigates computing students’ perceptions of ethical principles in AI, their self-assessed preparedness to apply them, and the presence of ethics-focused disciplines in undergraduate curricula. Method: We conducted a survey with 56 undergraduate computing students at the University of Brasília (UnB). The questionnaire included Likert-scale items on students’ familiarity with 21 AI ethical principles, their perceived importance, and whether these topics were covered during their studies. Descriptive statistics were used to analyze the results, and we discuss them in light of related literature. Results: Students overwhelmingly perceive ethical principles as important and report high agreement with their relevance to professional practice. However, a significant gap exists between perceived importance and curricular exposure: although most students agree on the need to learn ethical principles during their studies, a substantial number report not having had a dedicated course on the topic. Some principles, such as Privacy, Justice, and Transparency, were rated both highly important and well-understood, while others like Prosperity and Solidarity showed lower familiarity. Conclusions: The findings highlight the urgent need to integrate dedicated ethics courses into computing curricula. Ethical literacy is important not only for technical competency but also for responsible decision-making in AI development. Future work includes extending the study to other institutions and exploring pedagogical strategies for teaching AI ethics in computing programs.
Development and Evaluation of the Game LEAGUE OF QUALITY: ISO/IEC 25000 in an Educational Context
Context: Teaching software quality concepts can be challenging due to their abstract nature and limited engagement with traditional instructional methods. To address this, educational games have emerged as an effective strategy for improving motivation and learning outcomes, particularly in Information Technology programs. Aims: This paper reports on the development and evaluation of the educational game LEAGUE OF QUALITY: ISO/IEC 25000, which aims to teach software product quality concepts based on ISO/IEC 25000. The goal is to introduce diversity into the course and enhance students’ skills, engagement, and knowledge. Method: The development of the game followed four phases: pre-production, pilot testing, production, and finalization. The game was applied in classroom settings with IT students at a Brazilian university. Data was collected from both students and professors regarding their perceptions of the game’s effectiveness and usability. Results: Most students reported that the concepts presented in the game were applicable to real-world professional scenarios. They found the dynamics of the game clear, accessible, and engaging, contributing positively to their understanding. The game provided a better learning experience, especially for students with little prior knowledge. Professors also observed increased motivation, participation, and improved learning outcomes. Conclusions: LEAGUE OF QUALITY proved to be an effective and engaging educational tool for teaching software product quality. It enhanced student motivation, especially among those with limited prior knowledge, and promoted better understanding and retention of key concepts. Both students and professors reported increased engagement, with students highlighting the game’s relevance to real-world practice and its positive impact on the learning experience.
From Prompts to Prototypes: Exploring ChatGPT in Collaborative Software Prototyping Education
Generative AI (GenAI) tools, such as ChatGPT, have the potential to accelerate software prototyping and enhance team collaboration by generating content, expanding ideation, and enabling faster feedback cycles. As prototyping is central to agile and user-centered design, helping teams validate ideas, refine requirements, and align solutions with user needs, GenAI may further amplify these benefits through iterative and collaborative support. However, how to effectively integrate GenAI into collaborative prototyping remains underexplored in Software Engineering (SE) education. This study explores how ChatGPT supports collaborative prototyping among student teams, focusing on interaction, organization, and decisionmaking. We conducted a case study with undergraduate teams using ChatGPT in project-based prototyping. Data were gathered through a questionnaire, prompt analysis, and observation. Team collaboration and prior experience influenced the use of ChatGPT. Jointly crafted prompts led to more relevant outputs, while simpler prompts resulted in reliance on generic responses. Students valued the tool for speeding up prototyping and ideation, despite challenges with prompt refinement and occasional misalignment. The study provides insights into the use of GenAI for prototyping in SE education, shedding light on its benefits and limitations for co-creation and team dynamics.
A Qualitative Evaluation of an Experiment on the Application of Active Methodologies in Teaching Software Process Improvement
This paper presents a qualitative evaluation of an experiment involving the application of active methodologies to the teaching of Software Process Improvement (SPI). Recognizing that many academic courses cover SPI theoretically but inadequately prepare students for real-world challenges in the software industry, this work proposes the application and qualitative evaluation of a curriculum structured around active methodologies. The study is grounded in widely recognized models, such as the Capability Maturity Model Integration (CMMI) and the Brazilian Software Process Improvement Reference Model (MR-MPS-SW). The methodology used involved three main stages: mapping the essential SPI knowledge assets, conducting expert interviews, and peer review to validate the developed materials. An experiment was conducted with two groups of students to compare traditional teaching approaches with the proposed active methodologies. The qualitative results showed an expected improved performance in the experimental group, highlighting an improved ability to apply concepts in practice. Despite limitations such as the absence of subsequent evaluations in real professional settings and the requirement for continuous content updates, the study suggests that adopting active methodologies can effectively develop practical competencies that are crucial for future professionals.
Challenges in Teaching Aeronautical Software Quality Standards RTCA DO-178C and RTCA DO-200B
Teaching aeronautical software quality standards, such as RTCA DO-178C and RTCA DO-200B, presents significant challenges, mainly when delivered to students who do not work directly in this domain. This paper explores the complex characteristics of conveying these specialized standards to a diverse audience. We identify the primary challenges in the teaching-learning process through a detailed analysis of the difficulties faced. These challenges include the need to contextualize aeronautical quality standards with practical examples, the abstract nature of the content, and the necessity for interactive methods and teaching resources to facilitate comprehension. This study provides insights for educators and educational institutions to improve their approaches to teaching aeronautical software quality standards.
A Pedagogical Architecture for Systems Design and Development in Remote Learning Based on Software Ecosystems Characteristics
The COVID-19 pandemic forced universities to transition to remote learning, presenting challenges for teaching programming. Thus, it is important to teach programming through new system development trends, such as software ecosystems (SECO), that consider social and technical aspects to bring students closer to the reality of the industry, even remotely. This work aims to propose and apply PA4SDD-SECO, a pedagogical architecture for systems design and development in remote learning based on SECO characteristics. PA4SDD-SECO was applied in an undergraduate information systems course, where an observational study was used to analyze the relationship of influence among students, and a questionnaire was used to analyze both PA4SDD-SECO and the relationship of influence among students according to their perceptions. As a result, PA4SDD-SECO improved student cooperation, regardless of geographic distance, and helped share experiences. We found that feedback directed at one project in SECO can influence others, even when geographically distant.
Chatbot Development through Design Thinking Techniques: An Exploratory Study from Students’ Perspective
Text-based conversational agents (chatbots) have become central interfaces in modern interactive systems, widely used in contexts ranging from customer support to educational environments, personal assistants, and healthcare systems. However, developing chatbots is a challenging process, as it requires identifying requirements and understanding users’ perspectives on these systems. In this context, Design Thinking (DT) has emerged as a promising approach for promoting creativity, empathy, and innovation in developing solutions centered on the needs of end-users. Therefore, it is necessary to incorporate DT into the Software Engineering curriculum to provide students with the skills needed to develop chatbots effectively. In this sense, this paper presents an exploratory study conducted with 13 undergraduate students aiming to investigate the DT cycles, with an emphasis on techniques that promote empathy in the conception, design, and evaluation of chatbots. The study involved participants in practical immersion activities, utilizing several methods to understand the context of use and the needs of users. We collected the data through diaries and questionnaires answered by the participants. The results show that participants were able to identify latent needs, generate more empathetic and contextually relevant proposals, and engage more actively in the design process.
Fostering Creativity with ChatGPT in Requirements Engineering Education: An Academic Experience
In the context of Software Engineering education, the Requirements Engineering (RE) phase plays a key role, as it directly influences the quality of the final product. Among the key competencies to be developed in this process is creativity, which is essential for developing innovative solutions that meet user needs. Since the emergence of Large Language Models (LLMs), their use has become more widely adopted in various stages of software engineering (SE), including RE. Thus, stimulating creative thinking through the use of LLMs in the classroom is a promising strategy for preparing future professionals capable of facing complex challenges and proposing original solutions from the early stages of software development. Based on this, the present study investigated the relationship between the use of LLMs, specifically ChatGPT-3.5, and the stimulation of creativity during software RE. For this purpose, we conducted a study with 42 students enrolled in an introductory SE course. Participants did a creative requirements generation activity using the Brainwriting technique, conducted in two stages: the first without the support of tools, and the second with the use of ChatGPT. The requirements generated in both stages were evaluated based on four dimensions from the Torrance Test of Creative Thinking (TTCT): fluency, flexibility, originality, and elaboration. The results showed that using ChatGPT helped increase the number of ideas, the variety of categories covered, the level of detail, and, in many cases, the originality of the requirements. Despite this, it also highlighted limitations, such as the need for intensive filtering of the generated requirements and the superficiality of some proposals. These findings reinforce the potential of LLMs, when critically mediated, as a pedagogical tool to foster creative and reflective skills in the training of future software engineers.
Critical Failure: A Card Game to Support the Learning of Non-Functional Testing: Exploring Software Quality Attributes through Competition and Engagement
Teaching non-functional testing and software quality attributes presents persistent challenges due to the abstract and theoretical nature of the content. To address this, we developed Critical Failure, a card game designed to support learning through playful competition and active engagement. Inspired by poker mechanics and adapted for educational use, the game encourages players to evaluate and critique each other’s hands as software systems, simulating the evaluation of quality attributes such as performance, usability, and security. The game was created as part of a Software Testing course to offer a more interactive alternative to traditional lectures. The game was played by a group of 17 students who provided highly positive feedback on its clarity, engagement, and educational value. Suggestions for improvement were incorporated into the final version, refining both gameplay balance and alignment with course content. The activity demonstrated strong potential as a didactic tool, helping students not only recognize various non-functional requirements but also understand how they interact and impact the testing process. This article presents the design of the game and reports on its initial classroom use as an exploratory activity, intended to gather impressions, refine mechanics, and assess its potential as a complementary tool for teaching non-functional testing concepts.
Cognitive Mapping in Software Quality: A Theoretical Proposal for the Inclusion of People with ADHD
Studies indicate that approximately 15% to 20% of the global population may be considered neurodivergent, including individuals with Tourette Syndrome, Dyslexia, Autism Spectrum Disorder (ASD), and Attention Deficit Hyperactivity Disorder (ADHD). It is also estimated that around 7 million children between the ages of 3 and 17 (11.4%) have been diagnosed with ADHD. Despite recent advances in promoting inclusion within Software Engineering, there is still a lack of structured strategies to adapt the teaching and practice of Software Quality to the needs of neurodivergent individuals. This study proposes a mapping between one of the behavioral profiles of ADHD, as defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), and the main activities in the Software Quality domain, as outlined in the SWEBOK V4. A qualitative approach was adopted, based on a documentary review, in which the technical tasks described in the Software Quality area were cross-analyzed with the behavioral characteristics of ADHD. This process enabled the identification of potential cognitive barriers. As a result, a conceptual table is presented, relating technical activities, associated difficulties, and inclusive good practice guidelines, applicable both in educational and professional contexts. The proposed model contributes to cognitive inclusion in the field of Software Quality, supporting the development of strategies that enhance the engagement, learning, and performance of individuals with ADHD. Finally, it is suggested that future studies conduct empirical validations to deepen and apply the findings in real-world settings.
Teaching Requirements Engineering for Human-Centered AI: A Classroom Experience with the RE4HCAI Framework
Requirements Engineering for AI-based systems (RE4AI) demands approaches that integrate technical, social, and human-centered dimensions. For instance, when eliciting requirements for AI systems, one must account for data biases and imbalances that may lead to decisions reinforcing stereotypes and excluding certain user profiles without making such effects visible. Despite the topic’s relevance, teaching RE4AI remains nascent and lacks suitable pedagogical methods. To address this gap, this study examines the use of the RE4HCAI framework as a pedagogical aid for teaching requirements elicitation in AI systems, attending to technical, human, and social dimensions. We conducted a classroom activity with 86 students, applying the framework and collecting perceptions of its usefulness. Among participants, 66% reported that using the RE4HCAI framework fostered reflection on themes such as responsibility, bias, and explainability. Despite its formative potential, students noted challenges related to linguistic complexity, the need for prior knowledge, and the clarity of instructions. This work offers preliminary empirical indications and provides structured materials that can support instructors in teaching RE4AI.
Design Thinking as an Educational Tool for Project Management Classes
Design Thinking has gained wide acceptance in the software industry as an effective approach for fostering innovation and user-centered solutions. This paper presents the design and application of a classroom activity aimed at introducing Design Thinking within the context of a Software Project Management course. The activity engaged undergraduate students in Computer Science and Information Systems in the creation of educational game prototypes based on course topics, enabling them to experience each stage of the Design Thinking process. Due to time constraints in academic settings, we adapted the original method to fit within limited class sessions, resulting in a relevant outcome of this work. Through collaborative problem-solving and iterative prototyping, students explored alternative ways to approach technical content. The results indicate that the use of Design Thinking in this setting not only supported conceptual understanding and teamwork but also encouraged creativity and a stronger alignment between academic learning and industry-relevant practices.
Augmenting, Not Replacing: The Role of Generative AI in Teaching Software Modeling
Context: Software Modeling is a fundamental component of Computer Science curricula, addressing diverse themes including domain modeling. Despite its importance and years of research, a persistent challenge remains: students’ difficulty in comprehending and applying core modeling concepts. While numerous studies have identified these issues and proposed solutions, many involving collaborative and feedback-based environments, the recent advances of Large Language Models (LLMs) offer a new frontier. This study proposes and evaluates a pedagogical approach that leverages a generative LLM to improve students’ practical skills in modeling UML class diagrams. Method: In an empirical study with 30 undergraduate students, participants first created a class diagram for a given problem, then used an LLM for the same task, and finally compared both models. We analyzed quantitative pre- and post-study self-assessments and qualitative feedback on the process. Results: The study revealed a statistically significant improvement (p<0.05) in students’ self-assessed knowledge for class diagrams, an effect not observed in control topics. Qualitatively, students praised the LLM for accelerating ideation and identifying errors, but also noted significant inaccuracies in the generated models, particularly in class relationships. Students perceive the LLM not as an autonomous ’autopilot’ for generating solutions, but as a valuable yet flawed ’co-pilot’ that augments the learning process. This suggests LLMs are effective tools for support and validation rather than direct solution generation.
Smart Prediction for Test Smell Refactorings
Software plays a critical role in modern society, making software testing essential for ensuring quality. Poor testing practices, known as test smells, reduce test code quality and maintainability. Although refactoring is a widely used technique to address these issues, little is known about when developers choose to refactor test smells and whether these actions actually improve test code quality. This study introduces a machine learning approach to guide test smell refactoring. First, we aim to mine refactorings performed by developers to derive a catalog of test-specific refactorings. Findings show testing framework evolution helps address test smells. Second, we aim to understand whether developers target low-quality test codes to perform refactorings and their effects on code quality. Our findings reveal that developers tend to refactor structurally low-quality test code more often. Third, we aim to learn whether developers would perform refactorings and which refactorings they would apply to improve the test code quality. Results show that Support Vector Machine models predicted refactoring decisions with 30–100% accuracy. This paper is a summary of the full paper version [8] of the doctoral thesis [7].
Elixir is a modern functional programming language that is steadily gaining popularity in the industry. However, there is still limited research on the internal code quality produced using this language. To address this gap, this Ph.D. thesis explores code smells and refactorings specific to Elixir, taking inspiration from Fowler’s classic catalogs. The first two studies employed a mixed-methods approach to identify and catalog 35 code smells (including 23 new ones and 12 commonly known), validated by 181 developers, as well as 82 refactorings (14 of which are novel), validated by 151 developers. The third study mapped relationships between code smells and corresponding refactorings, proposing practical guidelines for their systematic elimination. These findings had a real impact on the community of developers working with the language, contributing both to the prevention of code smells and to supporting the prioritization of refactorings in Elixir development.
Open Innovation and CrowdRE for Requirements Change Management in Software Ecosystems
Software ecosystems (SECO) have introduced complexity in requirements change management due to their open and dynamic nature. In SECO, multiple actors collaborate across organizational boundaries and can form distinct crowds that communicate requirements changes. This work proposes a method to assist professionals in requirements change identification in SECO (SECO-RCI) based on open innovation and crowd-based requirements engineering (CrowdRE). We applied Design Science to build SECO-RCI and evaluated its acceptance via a focus group with experts and its feasibility via a case study in a large real-world scenario. The focus group indicated positive acceptance of SECO-RCI, highlighting its ease of use, usefulness, and intention to use. The case study revealed the absence of a standardized approach for requirements change identification and emphasized the importance of centralized visibility across feedback channels. These findings suggest that SECO-RCI assists in addressing key challenges of requirements change in SECO. Consequently, SECO-RCI can improve software quality in complex and distributed development environments, promoting transparent and collaborative management of evolving requirements.
On the Interaction between Software Engineers and Data Scientists when Building Machine Learning-Enabled Systems
Engineering ML-enabled systems presents various challenges from both a theoretical and practical perspective. One of the key challenges is the effective interaction between actors with different backgrounds who need to work closely together, such as software engineers and data scientists. This dissertation involved three studies investigating the current collaboration dynamics between these two roles in ML projects. Our studies revealed several challenges that can hinder collaboration between software engineers and data scientists, including differences in technical expertise and unclear definitions of each role’s duties. Potential solutions to address these challenges include encouraging team communication and producing concise system documentation.
Automatic Systematic GUI Testing for Web Applications
Graphical User Interface (GUI) testing plays a fundamental role in ensuring the reliability and quality of web applications by validating functionalities and exposing faults. Automated GUI testing tools are crucial for scaling this process, particularly in identifying visible failures such as crashes, error messages, and unexpected behaviors. However, their application in real-world, large-scale systems poses persistent challenges, including: (i) the automatic and non-redundant discovery of actionable elements, (ii) reliable synchronization with dynamic content, and (iii) efficient and thorough exploration of complex GUIs. This work addresses these challenges through the design and evaluation of three novel techniques: Unique Actionable Elements Search (UAES) for accurate element detection, the NetworkWait mechanism for robust synchronization, and the Iterative Deepening URL-Based Search (IDUBS) algorithm to guide scalable and effective exploration. These techniques are integrated into Cytestion, a new automated GUI testing tool for web applications. Cytestion follows a scriptless and progressive testing approach, starting from a seed test and incrementally exploring the interface by leveraging browser console errors, HTTP status codes, and GUI failure indicators as oracles. We evaluated our solution in a series of empirical studies involving four open-source and twenty industrial web applications. Results show that Cytestion significantly outperforms a state-of-the-art GUI testing tool in both fault detection effectiveness and runtime efficiency. In addition to its academic contributions, Cytestion has been adopted as part of the quality gate mechanism of an industry partner, demonstrating its practical applicability and impact.
Exploring Power Distance in Organizations Undergoing Agile Transformations
Power distance is defined as the degree to which members of a society expect and accept the unequal distribution of power among people. In organizations undergoing agile transformation, high power distance can result in restricted communication, reduced collaboration, and resistance to change—factors that can negatively impact software quality. However, identifying latent behaviors associated with this high power distance is not always straightforward. This paper presents how the Instrument for Diagnosing Power Distance (IDPD) was developed and evaluated in an industrial setting. The research method adopted was Design Science Research (DSR) and followed guidelines for the development of psychometric scales. The IDPD’s measurement items are based on manifestations of power distance related to human factors, elements of organizational culture, and process-related aspects identified through a systematic literature review. The instrument was preliminary evaluated by a panel of experts, applied in a case study with two agile organizations, and validated through a survey. The experts considered the IDPD items to be clear, relevant, and adequate for measuring power distance in agile contexts. The case study diagnosis indicated that high power distance had little to no effect on elements of organizational culture and human factors. However, process execution and improvement may be affected by high power distance. The participants considered the results relevant and indicated their intention for future use of the IDPD. The survey preliminarily showed good psychometric indices. Initial evidence suggests that the IDPD can be used to diagnose power distance in agile organizations or organizations undergoing agile transformation.
Functional vs. Object-Oriented: Comparing How Programming Paradigms Affect the Architectural Characteristics of Systems
This study compares the impact of adopting object-oriented programming (OOP) or functional programming (FP) on the architectural characteristics of software systems. For that, it examines the design and implementation of a Digital Wallet system developed in Kotlin (for OOP) and Scala (for FP). The comparison is made through a mixed-method approach. The self-ethnographic qualitative analysis provides a side-by-side comparison of both implementations, revealing the perspective of those writing such code. The survey-based quantitative analysis gathers feedback from developers with diverse backgrounds, showing their impressions of those reading this code. Hopefully, these results may be useful for developers seeking to decide which paradigm is best suited for their next project.
Fairness Testing in Retrieval-Augmented Generation: How Small Perturbations Reveal Bias in Small Language Models
Large Language Models (LLMs) are widely used across multiple domains but continue to raise concerns regarding security and fairness. Beyond known attack vectors such as data poisoning and prompt injection, LLMs are also vulnerable to fairness bugs. These refer to unintended behaviors influenced by sensitive demographic cues (e.g., race or sexual orientation) that should not affect outcomes. Another key issue is hallucination, where models generate plausible yet false information. Retrieval-Augmented Generation (RAG) has emerged as a strategy to mitigate hallucinations by combining external retrieval with text generation. However, its adoption raises new fairness concerns, as the retrieved content itself may surface or amplify bias. This study conducts fairness testing through metamorphic testing (MT), introducing controlled demographic perturbations in prompts to assess fairness in sentiment analysis performed by three Small Language Models (SLMs) hosted on HuggingFace (Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, and Llama-3.1-Nemotron-8B), each integrated into a RAG pipeline. Results show that minor demographic variations can break up to one third of metamorphic relations (MRs). A detailed analysis of these failures reveals a consistent bias hierarchy, with perturbations involving racial cues being the predominant cause of the violations. In addition to offering a comparative evaluation, this work reinforces that the retrieval component in RAG must be carefully curated to prevent bias amplification. The findings serve as a practical alert for developers, testers and small organizations aiming to adopt accessible SLMs without compromising fairness or reliability.
Exploring Code Clone Behavior in Modern Code Review
The Modern Code Review (MCR) process is iterative and asynchronous, enabling the early identification of issues during development. One of the main challenges in this context is the presence of code clones, fragments of code copied with small modifications that hinder maintainability. In this study, we analyzed 80k revisions from the CROP dataset using the Siamese detector, identifying 27,656 relevant clones across six systems. A manual validation indicated a predominance of Type-I (46.7%) and Type-III (45.3%) clones. We also identified 224 reviews in which clones appeared in a single revision (Single), 1,258 reviews in which clones appeared across multiple revisions (Recurring), and 236 reviews at the intersection of both categories. To deepen the analysis, we introduced two metrics, Duration and Distance, to assess how clones are introduced or removed during the review. This paper presents an expanded abstract of the study “An Exploratory Study on the lifecycle of Code Clones During Code Review”, published at SBES 2025.
Beyond Green Tests: Removing Smells From Natural Language Tests
Test smells signal design flaws in tests, harming maintainability and reliability. While automated test smells are well-studied, natural language test smells remain underexplored. Prior work identified 13 such smells but lacked systematic removal strategies and automated tools.We bridge this gap by presenting a catalog of transformations for seven key natural language test smells and a NLP-based tool for automated detection and correction. We evaluated our approach through a survey of 15 professionals and empirical analysis of Ubuntu OS test cases. Results show high professional acceptance (91.43%) and strong tool precision (83.70% F-Measure). Our work is the first to systematically address natural language test smell removal.