Proceedings of the Brazilian Symposium on Databases (SBBD)
Sociedade Brasileira de Computação - SBC
This document brings the Introduction and the Organizing Committee of the Proceedings of SBBD 2025.
Incremental and Risk-Sensitive Federated Learning for Ranking Models in Scenarios with Heterogeneous Data Distributions
This work proposes a new Federated Learning for Ranking (FL2R) strategy in scenarios with non-independent and non-identically distributed (non-IID) data across clients. We present FedRisk, a risk-aware aggregation method that weights client contributions according to their reliability, combined with a mechanism for reusing parameters from the previous global model, to mitigate the effects of data heterogeneity. Experiments with the MSLR-WEB10K ensemble show that FedRisk outperforms FedProx, the most robust baseline, by reducing the performance gap between federated and centralized models. FedRisk achieved a 15.6% improvement on nDCG@5 over FedProx and substantially reduced variance, increasing inter-round stability. Furthermore, for metrics such as nDCG@10, FedRisk matched the performance of the centralized model — a first among the compared methods, especially in a non-IID federated setting.
Exploring a New Metric for Accurately Measuring Blocking Precision in Entity Resolution Tasks
Entity resolution is a crucial task in data integration, aiming to identify records that refer to the same real-world entity. Blocking techniques are widely used to improve efficiency by reducing the number of record comparisons. However, traditional metrics, such as Pair Quality (PQ), fail to account for redundant comparisons, potentially distorting the assessment of blocking effectiveness. This paper introduces the PQ∗ metric, designed to provide a more accurate precision measure by eliminating the impact of redundant comparisons. We also propose the PQ∗C and P̂Q∗E algorithms, which efficiently calculate and estimate PQ∗, respectively. Experimental results show that, for all evaluated datasets, PQ and PQ∗ yield different results in every scenario where more than one blocking key is used. Furthermore, the difference between the two metrics increases as more blocking keys are employed for indexing.
Enhancing ML Explainability with Multi-Agent LLMs: A Context-Aware XAI Approach
Explainable AI (XAI) techniques enhance ML interpretability but often require technical expertise. We propose a multi-agent architecture that improves LLM-generated explanations through structured reasoning and contextual retrieval. Our system integrates web search, Retrieval-Augmented Generation (RAG), and XAI outputs via specialized agents. Experiments on the Adult dataset show that our approach outperforms standard LLM explanations by 7% in Context Awareness. Additionally, we validate LLM as a Judge, achieving over 80% correlation with human evaluations. Different LLMs, including OpenAI’s GPT-4o and GPT-4o-mini, were tested, highlighting the effectiveness of multi-agent systems in ML explainability.
An Architecture for Geospatial Big Data from IoT Devices with Heterogeneous Levels of Mobility
The growth of geospatial data in internet of things raises questions about the influence of device mobility on storage strategies. This work proposes an approach for geospatial big data from IoT devices with heterogeneous mobility levels, with two main contributions: i) a survey of the key technologies used for storing and processing geospatial big data, highlighting their characteristics and functionalities; and ii) a three-layer architecture for storing data from entities with varying mobility levels. A prototype of the architecture was implemented, and the data storage and analysis layer was tested in three scenarios simulating different levels of mobility.
Dataflow Analysis of Serverless Scientific Applications using Provenance Data
This paper presents an approach to ease dataflow analysis in Computational Science and Engineering (CSE) applications that invoke serverless functions. By capturing provenance data while executing CSE applications within a serverless environment, the approach organizes this information into an integrated database that users can query at runtime. Since serverless platforms typically lack native support for provenance tracking, the proposed solution helps users understand and analyze the behavior and outcomes of their CSE applications. We detail the main features of the approach as implemented in the DENETHOR tool and evaluate its effectiveness with a real-world CSE application. The results demonstrate that DENETHOR enhances analytical capabilities via its provenance database.
Use of Shadow Pipelines to Ensure ETL Pipelines’ Compliance with the LGPD
The General Data Protection Law (LGPD) defines guidelines for the collection, storage, and use of personal data in Brazil. Although there are solutions for adapting information systems to the LGPD, Extraction, Transformation, and Loading (ETL) pipelines, which are fundamental within organizations, still lack specific legal compliance solutions. This paper proposes an approach based on shadow pipelines to adapt existing ETL pipelines to the LGPD. The approach enables the verification of legal basis for processing provided for in its article 7. The proposal was evaluated through a feasibility study using synthetic pipelines. The results demonstrate that the approach supports LGPD compliance while preserving the original pipeline’s structure and performance.
Detecting Anomalous Vessel Trajectories: A Collaborative Clustering-Based Approach
Thousands of vessels operating worldwide may present issues related to anomalous trajectories, often characterized by unexpected changes in course, speed, or navigation through restricted areas. Manually detecting these anomalies is impractical, evidencing the need for automated support for maritime surveillance agents. Although Automatic Identification Systems (AIS) enhance situational awareness, they are insufficient for detecting trajectory anomalies. Existing approaches already use AIS data to detect anomalous vessel trajectories, but they do not consider contextual variables (e.g., oil spills) or the domain expertise of maritime surveillance agents. This paper introduces INSTRUCTOR, a solution that combines AIS data with clustering algorithms to group vessel trajectories based on similar directions and movement patterns. This clustering output assists experts in collaboratively identifying anomalous trajectories, a process validated through the collective judgment of multiple domain specialists. Experiments conducted in partnership with the Brazilian Navy corroborate the effectiveness of INSTRUCTOR in detecting anomalous maritime trajectories.
Exploiting Surrogate Submodular and Cost-Effective Lazy Forward Algorithms for Calibrated Recommendations
Recommendation systems are tools to suggest items that might interest users. These systems rely on the user’s preference history to generate a list of suggestions most similar to items in the user’s history, aiming for better accuracy and minimal error. Pursuing higher accuracy can lead to side effects such as overspecialization, reduced diversity, and imbalances in categories, genres, or niches. Thus, this work explores algorithms for selecting the items that form a calibrated recommendation list. The hypothesis is that calibration can positively contribute to fairer recommendations based on user preferences. We introduce a variation of an existing algorithm, which performs better than the baselines in a commonly used dataset in two different divergence measurements.
Analysis of online and offline classification algorithms for human activity recognition using IMU sensors
Physical activity monitoring through machine learning, using data collected from wearable devices equipped with motion sensors and vital signs monitoring, such as heart rate, temperature, and blood oxygenation, has gained significant attention in sports and medical fields. This advancement enables real-time performance tracking and early detection of motor conditions. While offline classifiers achieve high accuracy, they cannot adapt to novel motion patterns; online (incremental) learners overcome this limitation. Although there are online learning algorithms, their application to Human Activity Recognition (HAR) remains limited. The challenge for offline and online approaches is generalizing activity detection based on time-series segments, relying solely on sensor data without additional information. This study analyzes the performance of offline and online algorithms for HAR by applying segmentation and feature extraction techniques and evaluating the adaptability of incremental learning models over time. The research follows a quantitative and prescriptive approach, employing various classifiers to address the problem of HAR. The results highlight the performance of the offline Autogluon Predictor applied to the PAMAP2 dataset, which the five best algorithms nearly tied and achieved average scores of 94% accuracy, 93% F1-score, 94% precision, and 94% recall, followed by CIF with lower results. For online learning the algorithms XGBoost Incremental performed best with 78% accuracy, 77% f1-score, 81% precision and 78% recall. While Adaptive Random Forest and BiLSTM also nearly tied in the online setting, achieving lower results compared to XGBI, this study shows promising results for both scenarios. It highlights that determining the most suitable learning paradigm (online or offline) is a decision for the data scientist, guided by the particularities of the problem and the necessary dynamism required for the scenario. By exploring different learning approaches for HAR and evaluating their effectiveness, this research contributes to the development of more adaptive and personalized systems with applications in health monitoring, sports, and medical diagnostics, fostering advancements in continuous and adaptive user data analysis.
Heuristic-Guided Text-to-SQL Translation with LLMs: Optimizing Natural Language Interfaces for Relational Databases
Text-to-SQL mapping process plays a crucial role in enabling non-technical users to interact with relational databases using natural language. While Large Language Models (LLMs) have shown promising results in benchmark datasets, their performance in real-world settings often deteriorates. In this work, we propose a modular and adaptive agent that leverages the opensource LLM DeepSeek to perform translations with integrated feedback and query optimization. Our agent is prompt-engineered to refine query generation based on user or system-provided corrections. Using the TPC-H and Mondial benchmarks, as well as a real-world database, we demonstrate improved accuracy and execution efficiency, highlighting the impact of feedback loops and heuristic-based query rewriting.
Scaling Laws for Text-to-SQL: A Study on the Relationship Between Language Model Size and Performance
Although large language models achieve good results in the Text-to-SQL task, their high computational cost limits adoption by small businesses. This study evaluates the feasibility of small language models as an alternative, analyzing the trade-off between model size and performance for the open-source Qwen2.5, in variants from 0.5B to 32B parameters. Experiments were conducted on Spider benchmark and on a database with information about Brazilian companies, to evaluate the effectiveness of the approach in a real-world application context. Results show that the 3B model represents the best balance between cost and performance, but the 14B and 32B models, although more costly, offer superior performance.
Educational Inequalities in Brazil: A Clustering Analysis of Educational and School Performance Indicators
The Brazilian education system faces structural and socioeconomic challenges, reflected in unequal access to education and low academic performance rates, especially in vulnerable regions. Analyzing educational indicators helps identify structural changes in education, assess the effectiveness of implemented policies, and monitor the evolution of educational quality. This work employs the clustering of educational and school performance indicators to identify factors for educational inequalities in Brazil. Based on data from several educational indicators from 2015, 2019, and 2021 provided by INEP, it was possible to identify municipalities with more similar profiles. In addition, the temporal analysis of the clusters allowed us to understand the evolution of inequalities over the years, providing information that can be useful for formulating more effective public policies and strategically allocating resources.
Data Mining to Characterize Hypertensive Individuals with Cardiovascular Diseases in Brazil
This study applied data mining to classify healthy individuals and those with hypertension and cardiovascular diseases (HA + CVD) in Brazil, using data from the 2019 National Health Survey (PNS). Algorithms such as Decision Tree, Random Forest, and Naive Bayes were tested. The models performed similarly, with Random Forest achieving 97% accuracy and sensitivity in identifying healthy individuals. However, classifying HA + CVD cases was more challenging, with lower sensitivity, possibly due to the absence of formal diagnoses and lifestyle factors. The results highlight the importance of more detailed and longitudinal data to improve the identification of chronic diseases.
Data Governance in Systems-of-Systems: A Provenance-Data-Oriented Approach
The development of Systems-of-Systems (SoS), which integrate independent systems through clear data flows, has grown in recent years. Despite advantages such as reuse and resilience, SoSs face challenges in data governance, especially due to the lack of mechanisms to control the data lifecycle. In SoSs, data generated by one system is often used by other systems, making it difficult to ensure traceability, quality, and integrity from collection to storage. This article proposes PROVGov-SoS, a governance approach based on the management of provenance data. The proposal structures the information flow between systems, allowing users to understand the data lifecycle within the SoS. The approach was evaluated through a feasibility study in a real SoS, showing promising results.
Leveraging Large Language Models for Time Series Prediction on Low-Frequency Data
Time series prediction is critical in domains such as economics, industry, and agriculture. Real-world scenarios often involve challenges like low data frequency, high variability, and non-repetitive patterns. Traditional statistical models and machine learning approaches, including Long Short-Term Memory (LSTM) networks, underperform in low-data contexts due to overfitting risks, intensive training requirements, and the lack of benchmarks tailored to such scenarios. Large language models (LLMs) have emerged as tools for time series forecasting, leveraging their ability to generalize and capture temporal dependencies and patterns across datasets without requiring extensive task-specific feature engineering. This study investigates the potential of time series foundation models (TSFMs), specifically Lag-Llama and Chronos, on low-frequency datasets by comparing zero-shot prediction across one-step-ahead and multi-step-ahead approaches. Our findings evaluate predictive error, robustness, efficiency, and applicability, showing how TSFMs address these limitations and enhance forecasting in data-scarce scenarios.
Fine-Tuning Detection Criteria for Enhancing Anomaly Detection in Time Series
Anomaly detection is the problem of identifying observations that do not conform to the typical ones in a time series. Detection methods implicitly define detection criteria, such as deviation measures, filter thresholds, and candidate anomaly selection strategies. Choosing inappropriate criteria results in inaccurate outputs, generating spurious alerts or missing events. Adjusting these criteria is essential for monitoring systems. To address this challenge, this paper explores the fine-tuning of deviation measures, filter thresholds, and candidate selection strategies. Experimental results show that the proper choice of criteria significantly improves anomaly detection performance, often with greater impact than changing the detection methods.
Reducing Subjectivity in Identifying Variations in the Level of Financial Asset Values
Support and resistance represent price levels at which buyers and sellers have shown significant interest in an asset. As such, they can help traders and investors make decisions. However, identifying support and resistance can be subjective due to factors such as analyst interpretation, inference, and market psychology. This article proposes an approach to identify support and resistance levels automatically and objectively by applying Machine Learning techniques with minimal human intervention. The results obtained indicate a generalizable solution capable of adapting to different markets, time granularities, and analytical configurations.
Efficient Reuse of Metric Indexes for Multi-resolution Queries
Performing similarity queries using resolutions different from those used for the data employed during the index construction is a challenge for traditional Metric Access Methods (MAMs). Varying data resolution occurs, for example, in scenarios involving reduced data fidelity, such as when reducing data resolution for efficient network transmission. This work proposes a novel way to extend a MAM, using the Slim-tree as a case study, to efficiently and adaptively support queries over transformed data using indices built using only the original, full-resolution data. The proposed approach modifies the pruning heuristic to incorporate safe upper bounds on distances in the transformed domain, guaranteeing correct query results while preserving much of the pruning power of the original MAM. Experimental results on image datasets demonstrate substantial performance improvements over sequential search, even under conditions of high compression.
Enhancing Retrieval-Augmented Generation through Sequential Fine-Tuning of Small Language Models
Language Models (LMs) excel in general knowledge but often face challenges in specialized domains, where complexity and constant evolution pose additional obstacles. This study enhances the performance of Retrieval-Augmented Generation (RAG) systems for the Question Answering (QA) task through sequential fine-tuning of the RAG components, employing Small Language Models (SLMs). Our approach adjusts both the embedding model and the generative model using minimal computational resources and improves overall effectiveness compared to vanilla RAG. The proposed methodology, scalable and cost-effective, enables the practical application of RAG systems across different domains and tasks.
Too Many Imbalanced Classes? Don’t Classify – Rank! A Retrieval-Augmented Generation (RAG)-labels Based Approach for Multi-class Text Classification
This paper proposes an approach for multi-class text classification in scenarios of high semantic complexity and severe class imbalance, reformulating the task as a ranking problem. The technique exploits the ability of Large Language Models (LLMs) to semantically represent classes through RAG-labels–enriched descriptions generated via Retrieval-Augmented Generation. This reformulation reduces the lexical-semantic gap between texts and classes, allowing the fusion of dense and sparse rankers. Our solution presents gains of up to 70% in MacroF1 compared to state-of-the-art LLMs (e.g., LLama and DeepSeek), with an average cost reduction of up to 10 times.
LLM-Guided Autonomous Agent for News Extraction
Maintaining traditional rule-based scrapers is costly, as minor layout changes can break their selectors. This paper presents an autonomous agent that integrates a modular scraping pipeline with a Large Language Model (LLM) and dynamic prompt engineering to extract news without prior knowledge of HTML structure. The agent runs observe–plan–act loops, using GPT4 to decide actions (scroll, click, extract) until a stopping criterion is met. Experiments with 4.898URLs from Seade and 12 unseen portals showed average recall of 91% and precision of 95%, statistically matching the legacy scraper. The system sustained over 90% quality even on dynamic DOMs, enabling scalable media monitoring with minimal human input.
New Brazilian Database for Recommendation Systems of Scientific Papers
This work presents a new dataset for Scientific Article Recommendation Systems (SARS). In addition to being scarce, many existing datasets in the SARS context rely solely on co-authorship relationships as a relevance criterion, overlooking the importance of explicit user evaluations. To address this issue, we propose a new dataset with explicitly defined relevance labels, comprising over 2,000 researchers, 30 areas of knowledge, and approximately 71,000 associated papers. The work includes a characterization and evaluation of the proposed dataset, alongside other widely used datasets in the literature.
Data Repository for Open Science in the Amazon Region
The Amazon region is vital for climate regulation and biodiversity conservation, generating large volumes of environmental and atmospheric data. However, these data are often scattered, hard to access, and lack standardization, hindering reuse and collaborative research. Open-science initiatives require platforms that comply with FAIR principles, ensuring proper curation, persistent identifiers, and dissemination. This paper presents DataMap, a data repository specialized in managing and processing Amazonian data, supporting visualization, discovery, cataloging, and publication of datasets, with automatic DOI generation. The platform enhances accessibility and interoperability, enabling exploration through web interfaces and REST APIs.
KV-RAPTOR: Scalable Tree-Structured Retrieval with KV Cache Compression for Question-Answering Systems
This paper introduces KV-RAPTOR, a latency-optimized variant of the RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) pipeline for Retrieval-Augmented Generation (RAG). By integrating CacheGen, a compressed key-value (KV) cache reuse mechanism, into RAPTOR’s tree-based index, we demonstrate that it is possible to reduce generation latency without sacrificing answer quality. We evaluate our method on both English and Portuguese datasets, showing consistent reductions in time-to-first-token and end-to-end latency while preserving performance across diverse linguistic and retrieval contexts.
OndePublicar.com: A Collaborative Web Platform about Academic Events and Journals Accepting Paper Submissions
OndePublicar.com is a collaborative Web system that integrates dynamic information about academic events and journals, along with their Call for Papers (CFP). The system combines multiple bibliometric indicators (Qualis CAPES, SJR, SNIP), offers advanced search capabilities, and maintains a calendar of upcoming CFPs. It allows any user to suggest publication venues and CFPs, while accredited curators validate the entries. Approved entries are enriched with bibliometric data and indexed via Apache Solr, achieving a median latency of 0.47s under 10,000 concurrent queries. OndePublicar.com distinctive features include: (i) a unified repository combining a CFP calendar with national and global quality metrics of journals and events classified by field; (ii) an open, scalable architecture for venue and CFP discovery; and (iii) a collaborative curation model.
eTRUMiner: Mining Multivariate Temporal Rules from Heterogeneous and Incomplete Time Series
This paper introduces eTRUMiner, a novel algorithm for mining multivariate temporal association rules from heterogeneous time series datasets with missing observations. Our approach enables user-defined discretization, discovers frequent patterns, and outputs rules in short or detailed formats, thus enhancing interpretability. The results show that: (i) the choice of discretization method significantly influences rule relevance; (ii) eTRUMiner preserves the rules with high confidence in a dataset with up to 15% missing data; and (iii) the extracted rules can capture plausible causal dynamics. These findings demonstrate eTRUMiner’s robustness to incomplete data and its usefulness for exploratory analysis and forecasting in complex temporal domains.
Unveiling Power on Combining Prompt Engineering Techniques: An Experimental Evaluation on Code Generation
Prompt engineering techniques have seen a significant rise in research interest as a means to achieve satisfactory results without retraining Language Models. This work presents a set of experiments to analyze the power of a combination of prompts. To this end, it evaluates six prompt techniques, combining them to result in twelve experimental scenarios applied to Python code generation. Evaluation using BERTScore indicates that Role combined with RAG achieves the highest performance in code generation with 98% similarity. Skeleton-of-Thought and Self-Verification reveal promising opportunities for the design of prompt templates. Our findings contribute to unveiling the power of combining prompt techniques for current applications such as code generation.
Everything in the same place at the same time: Data tracking in Smart Cities through Provenance
Smart Cities (SCs) use data from various sources to improve quality of life and support public policies. Ensuring complete traceability of this data, from collection to usage, is essential for auditing and reliability. Provenance data helps represent this derivation path, but collecting it in SCs is complex due to the diverse ecosystem of programs and users involved. To address this challenge, the ProvInCiA framework was developed to capture provenance data in SCs. Using the concept of meta-dataflows, the framework integrates data transformations, forming a continuous derivation path. The effectiveness of ProvInCiAwas evaluated in a case study on flood monitoring.
Intervening in problematic data regions to improve machine learning models
Debugging machine learning models is essential to improving their robustness and performance. This work explores a data-driven debugging approach based on problematic data regions—subsets of the data where the model performs poorly compared to others. These regions often reflect problems such as class imbalance, bias, or unfairness. We propose to improve model performance by focusing primarily on the data, using a specialized algorithm to identify problematic regions in the datasets, and then applying targeted interventions. A common cause for poor performance is class imbalance within a problematic region. For such scenarios, we apply data augmentation in a controlled manner, avoiding excessive introduction of synthetic data. Our experiments demonstrate that it is possible to improve model performance by focusing exclusively on problematic data regions rather than the entire dataset.
Effectiveness of Small and Large Language Models for PL/SQL Bad Smell Detection
The quality of PL/SQL code is critical for enterprise systems built on Oracle Database, yet traditional quality-assurance methods struggle to uncover nuanced code smells and complex semantic flaws. This study evaluates the effectiveness of two large language models (GPT-4o, Gemini 2.0 Flash) and two small language models (GPT-4o mini, Phi-4) for detection of bad smells in PL/SQL code. Using a uniform prompt structure, each model was tasked with identifying bad smells in a curated dataset of PL/SQL snippets. Results show that effectiveness varied significantly across models and bad smell types. These findings offer practical insights to aid in selecting and leveraging language models for PL/SQL code analysis.
Mitigating the Impact of Non-IID Distributions in Federated Learning for Recommendation Systems
This work investigates the impact of non-independent and identically distributed data distributions (non-IID) on the performance of movies Recommender Systems (SR) using Federated Learning (FL). Through the collaboration between parts, FL allows the distribution of computational cost and expands the data diversity available to the training process, favoring the construction of SRs with good generalization capacity. In this context, and to analyze the effectiveness of the partial data sharing strategy, experiments were conducted in four scenarios using the dataset Movielens1M: (C1) IID data; (C2) non-sharing non-IID data; (C3) non-IID data with 1% sharing; and (C4) non-IID data with 5% sharing. Two federated paradigms were used: cross-silo and cross-device. Experimental results indicate that partial data sharing is a promising approach to mitigate the adverse effects of non-IID distributions in federated learning, incurring approximately 9,5% increases for NDCG and 0.046 drops in the MAE, for example. Thus, predictive performance, privacy, and computational cost are balanced.
TWIX: Balancing Fairness and Utility in Item Exposure for Recommendation Systems
Recommendation systems personalize user experiences but often introduce the “rich-get-richer” effect, where popular items dominate visibility. This bias disadvantages new items or those with few interactions, where historical data limits the exposure of new items. To address these challenges, fairness in exposure has been studied through balanced exposure, which ensures equal visibility but may reduce recommendation utility, and quality-weighted exposure, which prioritizes high-quality items. To mitigate the trade-off between fairness and utility, we propose TWIX, a method that applies quality-weighted and balanced exposures determined by a threshold. Our approach reduces popularity bias and cold start issues while maintaining recommendation utility.
A migration approach from relational databases to document-oriented databases supported by LLMs
Relational databases are widely used to store data from various applications. However, they have limitations in scenarios involving high scalability and semi-structured data, creating opportunities for the use of NoSQL databases. In such cases, approaches for migrating data between relational and non-relational databases are crucial. This paper proposes an approach that generates data migration routines, using LLMs (Large Language Models) to analyze metadata from the relational database and generate a target NoSQL schema based on a set of queries provided by the user. Experiments conducted to migrate a PostgreSQL database to MongoDB demonstrated the feasibility of the approach.
Advanced Chunking Techniques: a Novel Approach for Semantic Splitters
Chunking, the process of splitting large amounts of text into processable parts, is an essential but often overlooked step for multiple Information Retrieval and Vector Databases tasks. Traditional chunking techniques rely on fixed-length or syntactic structures, creating opportunities for more meaningful approaches. Semantic chunking is the process of dividing text based on meaning and context, ensuring each chunk represents a logical unit of information. This work proposes the Dual Semantic Chunker, which represents an advancement over existing chunking methods by taking a closer look at semantic representation. We compared multiple chunking methods, including both semantic and traditional techniques, and achieved improved retrieval.
Equitable Diabetes Diagnosis: Tackling Ethnic and Gender Disparities
Machine Learning (ML) has advanced disease diagnosis in healthcare, but raises fairness concerns, as model biases can perpetuate social inequalities. This study aims to evaluate and mitigate bias in diabetes diagnosis prediction models. We conducted experiments considering ethnicity and gender as protected attributes, evaluating bias using the fairness metrics Statistical Parity Difference, Equal Opportunity Difference, and Average Odds Difference. We applied the bias mitigation techniques Reweighing and Prejudice Remover, which showed improvements in fairness metrics, with a reduction in disparities between groups, while maintaining model accuracy. These findings reinforce the need to integrate fairness considerations into ML models for healthcare applications.
Language Models are not a Panacea: Combining them with Domain Knowledge and Efficient Indexes for Entity Linking
Language models enable cutting-edge solutions for many problems. However, they may not always be the best choice—at least not on their own—for certain tasks in specific contexts. In this paper, we propose a hybrid approach to entity linking (EL) that employs domain knowledge and efficient indexes for named entity recognition (NER), delegating only the disambiguation step (NED) to language models. We evaluated this hybrid approach on textual descriptions of invoice items from public medication purchases. The experiments showed that domain knowledge and indexes enabled efficient recognition of medications (NER), with accuracy superior to most state-of-the-art language models investigated and comparable to the GPT-4o reasoning language model. In addition, candidate medications recognized by our computationally efficient approach were disambiguated (NED) by GPT-4o with 90.55% precision.
Delimited Domain, Exposed Hate: Using Prompts to Identify Online Hate Speech with LLMs
With the expansion of social media in recent decades, online hate speech has emerged as a relevant social and technological issue. Automatically identifying this type of content is challenging due to the inherent subjectivity of the messages, as well as cultural and individual variations. Large Language Models (LLMs) have emerged as promising alternatives for this task, but their performance is sensitive to prompt formulation. This paper investigates the impact of event contextualization in prompts on the task of hate speech detection. Three progressively structured prompt versions were developed and evaluated on the Command-A, GPT-4o, and DeepSeek-V3 models. Using real user comments collected around the release of the film Emilia Pérez, the results show that including context about the event in the prompt positively influences model performance, with improvements of up to 16% in the F1 score.
Comparative Study of NoSQL Databases for Geospatial Time Series Management
This study evaluates the performance of the MongoDB and InfluxDB database management systems in handling spatiotemporal data from the Brazil Data Cube (BDC) project, with a focus on applications related to coffee cultivation. The data were processed and modeled for storage in each database, both deployed in Docker containers. The methodology included data load tests and queries simulating real-world scenarios: one retrieving a time series for a single pixel, and another querying a spatial region on a specific date. The results showed that MongoDB outperformed InfluxDB in data loading and spatial region queries, while InfluxDB achieved better performance in pixel-based time series queries. This work offers a practical analysis to support the selection of appropriate technologies for efficient spatiotemporal data management.
Data Dependent Itemset Mining Under Local Differential Privacy
Frequent itemset mining with Apriori-based methods is widely used to uncover patterns in large datasets. However, under the private setting of Local Differential Privacy (LDP), this task poses key challenges, such as domain scalability and user set-valued data. Prior works address these problems through fixed-size transaction reporting protocols but often rely on hard-coded parameters, overlooking complexities from differing dataset profiles. To address this limitation, we propose a data-dependent framework for identifying the top-k most frequent itemsets under LDP. Through initial metadata queries and the integration of an optimization problem, we significantly improve over prior work on both synthetic and real-world datasets.
A Comparative Evaluation between DepreBERTBR and Language Models for the Classification of Depressive Texts
Depression has incapacitated people around the world. Identifying signs of this disorder in textual reports, in environments where an individual feels more comfortable, may enable early medical interventions. The DepreBERTBR is a pre-trained language model with specialized content in the domain of depression, likely to be explored. This work presents an experimental comparative evaluation between DepreBERTBR and general-domain monolingual or multilingual language models, considering Brazilian Portuguese and the task of classifying depression into four classes (absent, mild, moderate, or severe). The results show that DepreBERTBR is competitive, matching other models in terms of F1-score.
Dynamic Spatiotemporal Chunking for Matrix Data Series
Spatio-temporal matrix data series typically involve huge volumes, making partitioning essential for efficient querying. Current methods use static partitioning either by space or by time. This paper introduces a dynamic partitioning system that splits data spatially and groups it temporally based on query context over regions of interest. It maintains fixed-size spatio-temporal blocks to minimize secondary-storage block accesses. Experiments on geospatial datasets show up to a 75% reduction in query time, demonstrating the approach’s effectiveness.
Dynamics of Far-Right WhatsApp Groups in Brazil: A Comparative Analysis Before and After the 2022 Election
This study presents an exploratory analysis of the activity of Brazilian far-right WhatsApp groups across two distinct periods, namely the pre-election (August 16, 2022 to October 30, 2022) and post-election phases (November 1, 2022 to February 15, 2023). Based on a dataset comprising messages shared in 301 distinct public groups, each period under investigation was represented as a complex network. The analysis of these networks included centrality measures, core-periphery decomposition and community detection. Additionally, polarity analysis techniques and spatial visualizations were employed to complement the study of the investigated groups. The results revealed an increase in the number of groups within the network, rising from 154 in the pre-election period to 192 in the post-election period. Although the number of communities also increased, network modularity decreased from 0.537 to 0.292, indicating greater interconnectivity among groups. The number of core groups in the network increased from 23 to 39 (+69.6%). Nine instances of migration from the periphery to the core were identified, along with five in the opposite direction. The most central groups differed between the two periods. Finally, the shared messages exhibited a slight increase in polarity.
Twinscie-Prov: Managing Provenance in the ML Lifecycle with Twinscie
The increasing complexity of machine learning (ML) applications necessitates systems that ensure traceability and reproducibility. This work introduces the Twinscie-Prov approach, an adaptation of the W3C PROV standard, to structure data and process provenance throughout the ML lifecycle within the Twinscie system. Provenance data is stored in the Neo4j NoSQL database, enabling complex queries and auditing. Preliminary studies indicate that, for queries involving navigation through dependency graphs, typical in provenance data, the Neo4j implementation is up to five orders of magnitude faster than log-based approaches.
QualiBus: implementing quality metrics for public transport data
The analysis of public transportation data is essential for planning actions aimed at improving the public transport system in large cities, reducing operational costs, and enhancing urban mobility and user satisfaction. However, analyzing data related to the movement of thousands of vehicles integrated with IoT devices presents challenges regarding the quality and reliability of the information obtained, due to the presence of inconsistencies, inaccuracies, duplicates and gaps. This work proposes a framework for implementing data quality metrics for bus transport, which was validated through a case study using bus movement data from four Brazilian cities.
Scalable privacy-preserving record linkage: Evaluating MultiBit tree indexing in Atyimo
Privacy-preserving record linkage (PPRL) indexing techniques typically organize Bloom Filters (BF) into data structures to reduce unnecessary comparisons. However, widely used solutions like Multibit Trees (MTB) often face scalability issues with large datasets or high-dimensional BFs, requiring parallel or distributed computation. This study explores the integration of the MTB algorithm into Atyimo, a publicly available Brazilian PPRL tool for merging large-scale administrative databases. We used both simulated and real-world data in our experiments to evaluate Atyimo’s effectiveness with MTB in linking routinely collected health records in Brazil. The results show that our Spark DataFrame-based solution builds robust index structures that preserve the linkage accuracy and significantly reduce execution time compared to the baseline.
Efficient Parallel Algorithm for Key-Value Sorting
This work presents an efficient parallel algorithm for sorting key-value pairs on multicore processors. The main strategy consists of dividing the input data into a large number of mini-partitions. This approach improves load balancing among threads, enables the construction of appropriately sized partitions, maximizes the throughput of the parallel sorting phase, and consequently enhances overall performance. Experimental results show that the algorithm achieves good speedup compared to the state of the art, both for key-value pairs and for key-only sorting. It also demonstrates good scalability with the increase in the number of threads, a feature that is essential for its application on modern processors with a high number of cores.
An Approach to Managing Heterogeneous Data Lineage
Audit and data governance in projects involving multiple sources require reliable traceability to ensure data quality and trust throughout their lifecycle. Data lineage is an essential tool to map the origin, transformations, and destinations of data. This work proposes an approach to define and implement the complete trajectory of data over time. It uses standardized metadata based on the Dublin Core, enriched to capture and persist the history of transformations. The approach involves modeling specific metadata to track ETL (Extraction, Transformation, and Load) operations and proves feasible in a project utilizing real-world data.
Training-Free Hybrid Evidence Retrieval for Question Answering: Dynamic Fusion of Knowledge-Graph Triples and Dense Text Embeddings
This work presents a sub-50ms, training-free retrieval pipeline that leverages a Neo4j knowledge graph and a ChromaDB vector index. Questions and passages are embedded with Sentence-BERT, and the retrieved entities seed a one-hop Cypher expansion in the knowledge graph. A transparent fusion based on Dice-Sørensen overlap ranks both passages and triples. On the WebQSP and CQA-12k benchmarks, this hybrid method achieves superior Recall@10, MRR, and nDCG@10 compared to BM25, graph-only, and vectoronly baselines. Requiring no learned parameters and running on commodity hardware, it offers a practical alternative to heavyweight neural re-rankers and a robust evidence layer for retrieval-augmented generation (RAG).
Database Modeling Automation from Natural Language Requirements
This paper proposes an approach to support relational database modeling through the automatic generation of Entity-Relationship (ER) diagrams from natural language requirements, leveraging Large Language Models (LLMs) combined with prompt engineering. The method extracts entities, relationships, and attributes from textual descriptions to produce visual ER diagrams. The evaluation was conducted in two phases: first, by testing different LLMs based on structured criteria; and second, by validating the results with students familiar with database modeling. The results indicate that, while challenges remain in handling cardinalities and nullable constraints, the generated diagrams generally align well with the original requirements. These findings reinforce the potential of LLMs to enhance conceptual database modeling.
Sentiment Analysis in Book Reviews Using the Gemini API for Personalized Recommendation
This study investigates the use of the Gemini API for sentiment analysis in book reviews, focusing on genre-categorized evaluations. The methodology involved extracting predominant emotions for use in personalized recommendation systems. Results showed effectiveness in detecting positive sentiments, especially in fiction. Emotion-based recommendations closely matched user preferences which demonstrates how sentiment analysis can enhance personalization in book recommendation systems. Challenges in less popular genres were also identified, highlighting the need for more adaptive approaches.
Constructing High-Performance Portfolios: Optimization of High-Volatility Asset Portfolios Through Return Forecasting Using CNN+BiLSTM
This paper proposes an innovative approach for optimizing portfolios of high-volatility assets, integrating return prediction with CNN+BiLSTM into the Sharpe Index Model. The aim is to adapt the Sharpe Index Model to the specific characteristics of the high-volatility asset market, characterized by high volatility, non-normal distributions, and strong correlations between assets. The methodology uses a model that estimates the average return and future variance of assets, providing more accurate inputs to the SIM and resulting in the Sharpe Index Model with Prediction. The results show that the Sharpe Index Model with Prediction outperforms the model without prediction, with an average accumulated gain of 68.88%, compared to -19.76% with the SIM over 75 days.
Data-level sampling for dealing with imbalanced datasets: better protection against membership inference attacks
The use of machine learning models trained on imbalanced datasets with sensitive information has raised privacy concerns. One significant threat is the Membership Inference Attack (MIA), which tries to figure out if a particular data point was included in the training set. This paper investigates whether algorithm-level cost-sensitive learning poses a greater privacy leakage risk than data-level sampling. We conducted experiments using two datasets: the UCI Adult dataset, which focuses on predicting income, and the APS dataset, which focuses on predicting scientific productivity. Our results indicate that models trained with cost-sensitive learning are more vulnerable to MIAs. This supports the hypothesis that correcting for imbalances at the algorithm level can reveal more private information.
Subnational Foreign Trade Data in Latin America: Diagnostic, Challenges, and Technical Pathways
National foreign trade data (a country’s exports/imports) is widely available, but demand for the subnational level (states, municipalities) is growing. This brings volume and variety challenges, due to granularity and differing data handling across countries. This work analyzes the conditions of Latin American subnational foreign trade data to propose technical pathways for integration and standardization aimed at analytical uses. A layered minimal Lakehouse architecture, centered on Data Exploration and Understanding is employed, and case studies (Mexico and Uruguay) are presented. Findings reveal heterogeneity in data collection, compilation and dissemination, highlighting the need for harmonization to enable comparative analysis.
Preliminary Ranking-Based Selection for Optimized Retriever Configuration in RAG Systems
Retrieval Augmented Generation (RAG) systems rely on efficient retrievers to fetch relevant documents, with their performance influenced by factors such as chunking methods and embedding models. These components determine how documents are segmented and semantically represented, directly impacting retrieval effectiveness. To enhance retriever performance, this study explores the construction of an optimizer capable of selecting the best configuration for document retrieval within a predefined solution space. The selection of technologies is a critical step, considering project constraints, query nature, and domain-specific requirements. A key challenge is filtering out unsuitable technologies while ensuring optimal performance.
Enhancing Text-to-SQL with In-Context Learning: A Multi-Agent Approach Based on CHESS
Text-to-SQL has gained increasing attention with Large Language Models (LLMs). While existing architectures have demonstrated the potential of multi-agent systems there remains significant room for improvement. In this work, we extend the CHESS framework by integrating In-Context Learning (ICL) techniques into the Candidate Generator module, evaluating three strategies: Zero-Shot, Few-Shot Learning, and Retrieval-Augmented Generation (RAG). We implement the system using GPT-4o, and perform experiments on the financial dataset from BIRD-SQL. Results show that Few-Shot Learning and RAG significantly outperform the standard approach. Compared to Zero-Shot (59.31% Execution Accuracy (EX), 0.412 ROUGE-1), RAG significantly boosted performance, increasing EX to 69.48% and ROUGE-1 to 0.652.
Application of the ARIMA Model in Vertica for Wind Speed Forecasting
This study presents an exploratory analysis of applying the ARIMA model directly within the Vertica database for wind speed forecasting. A dataset from the EOSOLAR project was used, containing vertical wind profile measurements from the coastal region of Maranhão, Brazil. The evaluation considered the RMSE (Root Mean Squared Error) and MAE (Mean Absolute Error) metrics across 105 trained models. The study investigated whether the in-database ARIMA approach in Vertica could provide efficient modeling for wind speed forecasting. The results showed that models with low complexity achieved good predictive performance.
RPAs and Data Lakes for Industry 4.0: A Case Study of Integrated Data Ecosystem
The use of RPAs has accelerated process automation in corporate environments but presents performance limitations when dealing with large volumes of data. This context reveals opportunities for improving the scalability and efficiency of such solutions. This paper proposes a distributed, modular, and loosely coupled Data Lake architecture for the collection, storage, and processing of heterogeneous legacy data. The solution leverages open-source tools such as Hadoop, Spark, and Airflow, organized into functional layers. A case study was implemented using production line data at a multinational electronics company, demonstrating the feasibility and benefits of the proposed approach.
How Culture Shapes Customers: A Cross-Continent Analysis of Apps Reviews Using NLP Techniques
Understanding customers’ feedback is essential for businesses to improve products and adapt to different markets. This study analyzes 100,000 app reviews from Uber, Instagram, and WhatsApp across six countries (Brazil, U.S., Australia, India, U.K., and South Africa) to assess whether user concerns are global or culturally influenced. Using BERTopic for topic modeling, zero-shot classification for topic assignment, and Aspect-Based Sentiment Analysis, we identify twelve key topics in reviews. While some topics are shared globally, others are country-specific. For example, Uber’s reliability was a major concern in South Africa and Australia, while Brazilian users discussed WhatsApp voice messages more frequently. These findings help businesses detect market-specific trends, benchmark competitors, and address regional needs strategically.
Theoretical Analysis of the Impact of Missing Data in Sensitive Attributes on the p%-rule Fairness Metric
Algorithmic fairness assessment has become a central topic in the analysis of automated decision-making systems. In this paper, we investigate how the use of imputation techniques to fill in missing sensitive data can affect the value of the p%-rule metric, depending on the degree of imputation error. We propose a formal mathematical model to quantify this impact, considering symmetric and bidirectional imputation errors. Additionally, we analyze the minimum number of manual corrections required to achieve a desired improvement in the fairness metric. The results provide a quantitative foundation for understanding the trade-offs between the cost of manual intervention and the quality of imputed data in algorithmic fairness analysis.
Agnostic Architecture for Transcription, Storage, and Analysis of Operational Audio in the Electrical Sector Using Cloud Services and Elasticsearch
This article presents a cloud-based solution for the transcription, structured storage, and analysis of audio data from power system operations. The developed tool leverages Elasticsearch for efficient storage and indexing of transcribed data and integrates with data science pipelines for semi-automatic labeling, operator evaluation, and visualization through Business Intelligence (BI). The proposed architecture is cloud-agnostic and has been validated in AWS, GCP, and Azure environments. Experiments demonstrate an average reduction of 83.3% in the time required to search for operational events, highlighting the solution’s potential for auditing, informational efficiency, and reliability in the power sector.
Assessing the impact of missing value mechanisms on anomaly detection in healthcare wearable data
Remote health monitoring using wearable devices has transformed healthcare by enabling continuous observation and early intervention. However, these systems frequently suffer from missing data, which can introduce bias and impair clinical decision-making. The uncertainty surrounding the cause of missing values further complicates data analysis. This paper investigates the impact of different missing data mechanisms (i.e., MCAR, MAR, and MNAR) in the healthcare wearable data anomaly detection task. Using heart rate and step count data from patients with respiratory illnesses, we assess the performance of an anomaly detection method under varying missingness conditions. Our findings demonstrate that more complex mechanisms (MAR and MNAR) significantly degrade detection performance, even at low missing rates, highlighting the importance of developing robust imputation strategies tailored to the nature of missingness.
When Errors Inform: Supporting Diabetes Diagnosis in High-Uncertainty Scenarios
We investigated the effectiveness of supervised machine learning methods in identifying individuals who may be undiagnosed or at high risk of developing Diabetes Mellitus (DM) in the context of private health insurance providers. The scenario is challenging: only indirect administrative data are available (such as the type and frequency of exams), without access to clinical results, along with low class separability and label uncertainty. We evaluated three classifiers (XGBoost, Random Forest, and Logistic Regression), achieving robust performance (Macro-F1 of 90.1%). Error analysis suggests that false positives may indicate undiagnosed cases, while false negatives may reflect inadequate clinical management.
Drift Detection for Identifying Training Patterns Prior to Performance Improvement in Runners
The growing popularity of running in Brazil has led to an increase in amateur runner participation, creating a demand for personalized recommendations to support these athletes. This study aimed to identify runners who showed performance improvements by detecting concept drift in their training performance time series. Training metrics were computed and compared between periods preceding performance improvements and declines. It was observed that, in cases of improvement, these metrics were higher compared to the periods preceding performance declines. These differences were statistically significant (p-value < 0.05), indicating which training patterns are associated with performance progression and representing a first step toward deeper insights.
Collaborative Classification for Object Labeling on Expansible Datasets
Streaming applications in video monitoring networks generate datasets that are continuously expanding in terms of data amount and sources. Thus, given the sheer amount of data in these scenarios, one big and fundamental challenge is how to reliably automate data annotation. In this work, we propose a novel active learning strategy based on multi-model collaboration able to self-annotate training data providing only a small initial subset of human verified labels, towards incremental model improvement and distribution shifts adaptation. To validate our approach, we collected approximately 50,000 hours of video data sourced from 193 security cameras from University of São Paulo Monitoring System (USP-EMS) during the years 2021-2023, totaling 7.3TB of raw data. For experimental purposes, this work is focused on identification of pedestrians, cyclists and motorcyclists resulting in 3.5M unique objects labeled with accuracy between 92% to 96% for all evaluated cameras.
Evaluating the Limitations and Potentials of the k-Nearest Neighbors (kNN) Algorithm in Clinical Data Imputation
Data quality is crucial for the effectiveness of Machine Learning solutions in healthcare, with missing values being a common and critical issue. This study compares the MissForest and MICE imputation methods applied to clinical data from over 16,000 COVID-19 patients, highlighting the superior accuracy of MissForest, associated with high computational cost. We propose a KNN-based imputer optimized for this context. Although less accurate, its computational efficiency was significantly better. An analysis showed that KNN performance is influenced by neighborhood similarity and local entropy. In homogeneous scenarios, KNN proved competitive, suggesting its potential for hybrid imputation strategies that combine efficiency and robustness.
Probabilistic Representation of Vehicular Trajectories as Input for Artificial Neural Networks
This work proposes a probabilistic approach to represent vehicular trajectories using data from license plate recognition (LPR) cameras, aiming at real-time processing. It is based on the Hierarchical Pattern Bayes (HPB) model to generate matrices of trajectory typicality and temporal density. These matrices were evaluated using the Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) algorithm, resulting in a 75% increase in the Silhouette score and a 31.1% reduction in the Davies-Bouldin index compared to the use of raw coordinates. These results indicate a more structured representation, enabling the application of supervised models for real-time anomaly detection.
Metadata Management in Data Mesh: Toward Federated Discovery and Governance
Organizations striving for data-driven operations often encounter limitations with monolithic data architectures and centralized data teams, typically in terms of scalability and data governance. The Data Mesh paradigm has emerged as a promising, decentralized alternative, yet practical guidance for implementing its key components, particularly federated metadata management, remains limited. This paper systematically reviews academic contributions on metadata solutions in Data Mesh environments, analyzing architectural patterns, standards, and technologies. We investigate existing approaches, provide a detailed comparison, and pinpoint critical open challenges to enable scalable and interoperable metadata governance across domains.
Data Augmentation for Medical Image Segmentation: A Comparative Analysis of Traditional Techniques and Synthetic Data Generation
Deep Learning has been widely applied to medical image segmentation, aiming to make structures clearer in images to help physicians identify unusual patterns and anomalies. Segmentation models face challenges in collecting a large amount of data for training, due to privacy concerns and pathological representation. Data Augmentation (DA) is an alternative to mitigate this challenge, expanding the dataset by applying transformations to the original set or creating new samples using generative methods. Despite the extensive use of DA techniques, there is still limited understanding of their relative effectiveness for medical image segmentation tasks. This work presents a method for evaluating the impact of DA methods, analyzing traditional augmentation techniques, and diffusion models for generating synthetic data in medical image segmentation models.
Study of the Impact of Synthetic Data and Paraphrases on Mitigating Imbalance in Low-Sample Portuguese Text Classification Tasks
Class imbalance poses a major challenge in text classification, particularly in cenarios of scarce annotated data and low-resource languages like Portuguese. This study addresses the classification of technical support requests from healthcare professionals on the use of the Electronic Health Record in Primary Health Care of the Brazilian Unified Health System (e-SUS APS), based on a small and highly imbalanced dataset. Six resampling strategies were evaluated, including paraphrase-based data augmentation using Large Language Models. Combining paraphrasing with selective labeling improved BERTimbau’s Macro-F1 by 18%, matching the performance of Logistic Regression (LR) with random oversampling and selective labeling, which yielded a 70 improvement over the original method. However, LR is approximately 3690 times more efficient than BERTimbau when considering the most effective version of each method.
ARANI: An Experiment Line-Based Approach for Privacy Preservation in Data Lakes
Data Lakes store large volumes of heterogeneous data, including sensitive information. Ensuring compliance with regulations such as the LGPD requires the use of anonymization techniques. Techniques applied in isolation, such as k-Anonymity or Differential Privacy, may be insufficient. Therefore, combining these techniques in configurable flows is essential. Experiment Lines enable the flexible structuring and instantiation of these flows. This paper proposes ARANI, an Experiment Line-based approach that allows the definition, execution, and evaluation of anonymization flows with support for multiple techniques.
Quantifying the RAG Advantage: A Multi-Metric Benchmark for LLM-based Code Generation
The recent advancement of Large Language Models (LLMs) has demonstrated remarkable capabilities in solving programming challenges. However, despite their proficiency, LLMs often suffer from hallucination and limited performance on unfamiliar or complex tasks. Retrieval-Augmented Generation (RAG) has emerged as a promising solution to address these limitations by supplementing prompts with relevant external information. In this paper, we propose a benchmark to assess the efficacy of RAG in solving algorithmic problems by integrating a curated database of 120 LeetCode problems, each paired with corresponding solutions and explanations. An Information Retrieval (IR) system was employed to construct enhanced prompts for solving novel problems.
Towards Enabling the Analysis of Visual Exploration Processes through Interaction Provenance
The rapid growth of data has made accessing, integrating, and analyzing information increasingly challenging. While large-scale systems support processing and querying, interactive visualizations are essential for exploring complex datasets. Understanding how users gain insights from these visualizations requires capturing their interactions. Provenance data offers a natural solution, but current methods often fail to capture interaction-level provenance effectively. This paper presents an approach to capture and record user interaction provenance and integrate it with both prospective and retrospective provenance. We implement this approach in the Curio framework, which builds urban data visualization pipelines. Results demonstrate its effectiveness in capturing user behavior during visual exploration.
A Data Augmentation and Validation Pipeline for Improving Emotion Classification in Mobile App Reviews
This paper examines GPT-2 based data augmentation to improve sentiment classification of Portuguese mobile app reviews, employing the BERTimbau model. A pipeline is proposed, integrating synthetic data generation with semantic analysis, UMAP for dimensionality reduction, and HDBSCAN for clustering and validation. Results show validated and balanced synthetic augmentation boosts model performance in sparse or imbalanced data scenarios.
DiveScan: A Module for Data Retrieval by Similarity with Diversity in Postgres
Similarity queries with diversity are useful for applications that require results avoiding redundancy, such as medical diagnosis, recommendation systems, and image retrieval. However, the Database Management Systems still offer limited support for this type of query. This work presents the DiveScan module for Postgres, which executes such queries using the FM or BRID algorithms, extended to use GiST indexes to speed up the searches. Experimental evaluation shows that the indexed approach significantly outperforms sequential scanning in various scenarios, substantially reducing execution time.
Subdomain Identification Strategies for Efficient Machine Learning Models
The performance of machine learning models depends on both the quality of the data and the selection of the models. This work highlights the importance of identifying regions in the input space with similar behavior to improve predictive accuracy. Clustering techniques can partition multivariate time series (MTS) into meaningful subsets, enabling the training of specialized models. We employ k-Medoids and quadtree-based clustering, both using dynamic time warping (DTW) as a similarity measure, and the quadtree also incorporates entropy for partitioning. Long Short-Term Memory (LSTM) networks are trained on these clusters and compared to a global model trained on the entire dataset. The results support the subset modeling hypothesis, showing that models trained in clusters can achieve comparable performance to a global model. This approach offers a comparable alternative that balances prediction accuracy with computational and interpretable advantages.
UniChat: Architecture and Evaluation of a Low-Cost RAG Agent for the University Context
This paper introduces UniChat, a low-cost chatbot based on RAG to optimize access to university information. The solution utilizes free and open-source technologies like n8n, PostgreSQL, and Google Gemini. Its main contributions include the low-cost architecture , empirical analysis of the correlation between source document format and response accuracy , and validation of the user experience with Think Aloud and AttrakDiff Mini. Tests with real users demonstrated a 76.7% success rate and a positive perception of functionality and intuitiveness. UniChat has the potential to enhance university communication and reduce administrative overload.
Large Language Models for Bid-Rigging Detection in Public Procurement
Identifying collusion in public procurement remains a persistent challenge, with various machine learning-based approaches being explored in the literature. This study investigates the application of Large Language Models (LLMs) in detecting collusion in procurement processes. We conducted experiments using prompt engineering and fine-tuning techniques, comparing the performance of the models with that of traditional algorithms such as Random Forest, Logistic Regression, and Support Vector Machine. The results showed that, although prompt engineering did not yield satisfactory outcomes, finetuning enabled the ChatGPT 4o Mini to outperform traditional algorithms on the analyzed datasets.
Adapting Indicators for Assessing the Health of Scientific Infrastructure Projects: A Business Intelligence Application in Sirius Beamlines
This paper presents the development of a Business Intelligence (BI) tool aimed at monitoring the health of scientific infrastructure projects, in the context of the construction of the Sirius Particle Accelerator Beamlines. The solution is based on the implementation of key performance indicators (KPIs), which are currently being validated, with the aim of continuously reflecting factors such as changes in scope, pace of execution of activities and team engagement in updating schedules. The proposal seeks to contribute with quantitative approaches for the dynamic evaluation of projects with a high degree of uncertainty and variable scope, common characteristics of projects in the area of research and development (R&D).
PANDORA: Statistical System for Forecasting Extreme Weather Events
This paper highlights the importance of extreme weather forecasting systems and presents Pandora, a forecasting tool for extreme weather events. Pandora uses the Emergency Events Database (EM-DAT), focusing on the categories of extreme weather events relevant to the Brazilian territory. Preliminary results show that Pandora is a valid tool for forecasting and correlating data from extreme events in Brazil.
Using Apache Superset for Scalable Visualization of Public Education Data: A Case Study with the School Census
Public databases, such as the School Census provide data that facilitates the planning of educational policies in Brazil. Although the data are provided in an accessible format and tabular structure, the lack of adequate tools limits their exploration. This work evaluates the use of Apache Superset as an open-source solution for interactive visualization and creation of indicators in education. The data were loaded into a relational database and underwent a data ingestion and transformation flow. Superset was used to create educational indicators, with the aim of demonstrating how the tool can facilitate the analysis of temporal, spatial, and demographic patterns, supporting decision-making and the production of scientific knowledge.
A Comparative Study of Example Selection Strategies for In-Context Learning Applied to Automatic Text Classification with Large Language Models
Text Classification (TC) with Large Language Models (LLMs) can be performed via zero-shot (low cost, limited effectiveness) or fine-tuning (high cost, high effectiveness). This study explores example selection strategies for the intermediate in-context learning approach, comparing random selection to methods based on vector representations such as TF-IDF, SBERT, and LLM2Vec. Results confirm that fine-tuning Small Language Models (SLMs) offers the best cost-effectiveness trade-off. However, in-context learning emerges as a promising alternative, outperforming zero-shot without the fine-tuning computational cost, particularly when using effective selection strategies.
Weight and Expand: Impact and Limitations of Contextual-Sparse Representations in Topic Modeling
This work proposes using contextual-sparse representations in Topic Modeling (TM), aiming to combine the interpretability of sparse representations with the semantic power of contextual ones. Using the SPLADE model, we seek to represent documents in a context-sensitive manner through term expansion and weighting. We empirically evaluate this approach in comparison with other representations. The results indicate that term weighting enables effective TM, while term expansion, although promising, presents limitations due to the mismatch between the representation’s vocabulary and the original texts.
Automatic Feature Extraction from Acoustic Emission Signals Using Autoencoder Neural Networks for Pipeline Integrity Prediction
Acoustic Emission (AE) signal analysis is a widely used technique for monitoring structural degradation, such as wall thinning in pipelines. However, extracting informative features from raw AE signals remains challenging due to noise and the complexity of signal propagation. This work proposes a novel feature extraction approach using convolutional Denoising Autoencoders to reduce noise and generate compact, informative latent representations. The proposed method outperforms traditional techniques in thickness prediction tasks, particularly in high-noise scenarios with both synthetic and real data.
A Comparative Analysis of Denoising Methods for Deep Learning-Based Audio Event Detection in Noisy Agricultural Environments
This study investigates whether traditional denoising improves deep learning for Audio Event Detection (AED) in real-world, noisy environments like livestock farms. We evaluated three denoising algorithms — Spectral Subtraction, Adaptive Kalman Filter, and SD-ROM — on the aSwine dataset using state-of-the-art models, including Pretrained Audio Neural Networks (PANNs) and the Audio Spectrogram Transformer (AST). Contrary to conventional wisdom, all denoising methods proved detrimental, with a worst-case scenario showing a 65.8% mean Average Precision (mAP) decrease. We conclude that powerful models learn to be inherently noise-robust, making robust architectures a superior strategy to noise preprocessing.
Extending Continuous Range Queries to Metric Data
Several studies highlight the importance and complexity of determining the nearest points along a trajectory through a technique known as Continuous Query. Existing solutions for answering such queries rely on Euclidean coordinates in spatial data to verify whether the object of interest lies within the coverage of the route. However, this approach does not apply to purely metric data, as such data cannot be represented in an n-dimensional space. On the other hand, a distance function can be used to infer line segments over the metric space, similar to the association of a geometry. In this context, the present work associates Euclidean properties with the metric space to define the relationship between points and lines, extending continuous range queries to this data domain. The experiments demonstrate that this association enables efficient continuous range query on purely metric data, addressing the gap left by existing techniques in the literature.
Entity Matching with Large Language Models: comparative study with Entity Blocking approach
Entity Matching is essential to integrate data from different sources that refer to the same entity. Although pre-trained models that adopt Entity Blocking techniques are widely used in this task, the advancement of Large Language Models (LLMs) suggests new possibilities. This work compares Ditto, which applies optimization techniques to traditional models, with Orca2, an LLM based on Llama2 focused on reasoning. Despite its inferior initial performance, Orca2 demonstrates competitive potential, especially with future computational improvements. Thus, we seek to evaluate the feasibility of LLMs in Entity Matching, analyzing accuracy and computational cost.
Exploring Temporal Text-to-SQL Challenges in Brazilian Portuguese: Lessons from Educational Data
Recent advances in natural language processing have enabled translating natural language into SQL, but challenges remain in multilingual and temporal contexts. This short paper presents a beginner-level exploratory analysis of a prompt engineering strategy for text-to-SQL generation over Brazilian Portuguese educational data. Through 10 representative examples reflecting how real users might ask questions about open government data, we show how language variability, implicit temporal references, and mismatched expectations affect SQL generation and the reliability of standard evaluation metrics. This work contributes to historical data querying research and highlights persistent challenges for multilingual text-to-SQL systems.
Distributed Pipeline for Large-Scale Spatial Analysis: Evaluation of the 3 Rule of the 3-30-300 Index in Fortaleza Using Apache Spark and Sedona
The application of spatial operations to large datasets faces limitations in traditional geoprocessing tools and libraries such as Geopandas. This work presents a distributed pipeline based on Apache Spark and Sedona to analyze geolocated data of buildings and tree coverage in Fortaleza, Brazil, identifying residences with visibility of at least 3 trees within 30 meters. Batch processing, spatial indexing, and structured persistence enabled overcoming Python’s performance bottlenecks. Detailed documentation of modular code is provided, allowing scripts to be tracked and replicated in other cities to perform large-scale geometric and spatial operations.
Addressing Database-Related Issues in Digital Social Network Data Analysis
The analysis of vast Digital Social Network datasets often neglects database systems, compromising effectiveness and efficiency, particularly in qualitative studies. We present some preliminary practical experiments, including quantitative mention counting and qualitative topic identification, showcasing (i) that managing large-scale social data is relevant and (ii) how leveraging DBMS could yield more robust and insightful social network data analysis.
Deep Learning for Urban Flood Prediction: An LSTM Model Integrating Satellite Reanalysis and Historical Weather Data in Curitiba
Urban floods pose increasing threats, causing significant human and economic losses. This research focuses on improving flood prevention using deep learning techniques together with satellite reanalysis data. We present the results of the implementation of an LSTM Neural Network with satellite reanalysis data to predict floods in the city of Curitiba, Brazil. To assess the benefits of the proposed model, we compare the results with datasets from previous related models. The results of each trained model are analyzed based on Receiver Operating Characteristic (ROC) curves. The results show the positive impact of the use of satellite data with respect to the area under the curve (AUC) metric.