Proceedings of the Brazilian Conference on Intelligent Systems (BRACIS)
Sociedade Brasileira de Computação - SBC
A Contrastive Objective for Training Continuous Generative Flow Networks
Generative Flow Networks (GFlowNets) are a novel class of flexible amortized samplers for distributions supported on complex objects (e.g., graphs and sequences), achieving significant success in problems such as combinatorial optimization, drug discovery and natural language processing. Nonetheless, training of GFlowNets is challenging—partly because it relies on estimating high-dimensional integrals, including the log-partition function, via stochastic gradient descent (SGD). In particular, for distributions supported on non-discrete spaces, which have received far less attention from the recent literature, every previously proposed learning objective either depends on estimating a log-partition function or is restricted to on-policy training, which is susceptible to mode-collapse. In this context, inspired by the success of contrastive learning for variational inference, we propose the continuous contrastive loss (CCL) as the first objective function natively enabling off-policy training of continuous GFlowNets without reliance on the approximation of high-dimensional integrals via SGD, extending previous work based on discrete distributions. Additionally, we show that minimizing the CCL objective is empirically effective and often leads to faster training convergence than alternatives.
A Data Distribution-Based Ensemble Generation Applied to Wind Speed Forecasting
Variability in wind intensity and direction causes unstable electricity supply to the power system, making integrating this energy into the electrical system a significant challenge for operations and planning practices. Ensembles can be used as an alternative to address the complex patterns over time in wind speed time series. The appropriate size of the training partition for ensemble models depends on dataset characteristics. And, to enhance model training efficiency, it is important to minimize repetitive data. By maintaining a concise training set, it becomes easier to meet the requirements of software and hardware constraints. Mainly because, there is a growing interest in deploying machine learning models on edge devices. For these reasons, a new method called Local Distribution (LocDist) has been introduced in this paper to predict wind speed, utilizing local pattern recognition based on data distribution. In testing with three wind speed time series, LocDist created a compact training subset with less than 20% of the training data. The Diebold-Mariano hypothesis test was employed to assess the significance of the forecast errors of the proposal compared to individual and bagging methods that use the entire training set. The LocDist method with long short-term memory (LSTM) and gated recurrent unit (GRU) won in 100% of the cases. Additionally, the LocDist with extreme learning machines (ELM) and autoregressive integrated moving average (ARIMA) won or tied in 83% and 66% of the cases, respectively.
A Large Dataset of Spontaneous Speech with the Accent Spoken in São Paulo for Automatic Speech Recognition Evaluation
We present a freely available spontaneous speech corpus for the Brazilian Portuguese language and report preliminary automatic speech recognition (ASR) results, using both the Wav2Vec2-XLSR-53 and Distil-Whisper models fine-tuned and trained on our corpus. The NURC-SP Audio Corpus comprises 401 different speakers (204 females, 197 males) with a total of 239.30 h of transcribed audio recordings. To the best of our knowledge, this is the first large Paulistano accented spontaneous speech corpus dedicated to the ASR task in Portuguese. We first present the design and development procedures of the NURC-SP Audio Corpus, and then describe four ASR experiments in detail. The experiments demonstrated promising results for the applicability of the corpus for ASR. Specifically, we fine-tuned two versions of Wav2Vec2-XLSR-53 model, trained a Distil-Whisper model using our dataset with labels determined by Whisper Large-V3 model, and fine-tuned this Distil-Whisper model with our corpus. Our best results were the Distil-Whisper fine-tuned over NURC-SP Audio Corpus with a WER of 24.22% followed by a fine-tuned versions of Wav2Vec2-XLSR-53 model with a WER of 33.73%, that is almost 10% point worse than Distil-Whisper’s. To enable experiment reproducibility, we share the NURC-SP Audio Corpus dataset, pre-trained models, and training recipes in Hugging-Face and Github repositories.
A Multi-level Semantics Formalism for Multi-Agent Microservices
Operational semantics is a fundamental approach to the formalisation of programming languages and almost a standard when it comes to agent-oriented programming languages. It helps ensure the correctness of interpreters, facilitates their implementation, and supports proofs of important properties. However, in the literature, there are few attempts of formalising multi-level operational semantics for multi-agent systems which allows the formalisation of all levels of abstraction in those systems. Also, recently, authors have started to discuss a new approach to build multi-agent systems known as multi-agent microservices, arguing that microservices represent a potential point of convergence between modern software engineering and multi-agent systems. We agree that microservices could be a useful approach to think of multi-agent systems and, in this paper, we propose a multi-level semantics to formalise this new approach to build multi-agent systems based on the abstraction of microservices. We not only introduce a template for those formalisation but we also show how our operational semantics style promotes desired characteristics of modularity, reusability, and a clear specification of shared communication artifacts in multi-agent microservices.
A Novel Genetic Algorithm Approach for Discriminative Subspace Optimization
Image set representation by subspace methods has shown to be effective for several image processing tasks, such as classifying multiple images and videos. A subspace exploits the geometrical structure in which images are distributed, representing the image set with a fixed dimension giving more statistical robustness to input noise and compactness to the images. The mutual subspace method (MSM) and its extensions, the Orthogonal Mutual Subspace method (OMSM), and the Generalized Difference Subspace (GDS) are the most prominent subspace methods employed. However, these methods require solving a nonlinear optimization which lacks a closed-form solution. In this paper, we present a metaheuristic-based approach for discriminative subspace optimization. We develop a Genetic Algorithm (GA) for integrating OMSM and GDS discriminative subspaces. The initialization strategy and the genetic operators of the GA provide quality of objective function value of solutions and preserve their feasibility without any extra repair step. We validated our approach on four object recognition datasets. Results show that our optimization method outperforms related methods in accuracy and highlights the use of evolutionary algorithms for subspace optimization. Code: https://github.com/bernardo-gatto/Evolving_manifold.
A Performance Increment Strategy for Semantic Segmentation of Low-Resolution Images from Damaged Roads
Autonomous driving needs good roads, but 85% of Brazilian roads have damages that deep learning models may not regard as most semantic segmentation datasets for autonomous driving are high-resolution images of well-maintained urban roads. A representative dataset for emerging countries consists of low-resolution images of poorly maintained roads and includes labels of damage classes; in this scenario, three challenges arise: objects with few pixels, objects with undefined shapes, and highly underrepresented classes. To tackle these challenges, this work proposes the Performance Increment Strategy for Semantic Segmentation (PISSS) as a methodology of 14 training experiments to boost performance. With PISSS, we reached state-of-the-art results of 79.8 and 68.8 mIoU on the Road Traversing Knowledge (RTK) and Technik Autonomer Systeme 500 (TAS500) test sets, respectively. Furthermore, we also offer an analysis of DeepLabV3+ pitfalls for small object segmentation (Code available on https://github.com/tldrafael/pisss).
A Unified Framework for Average Reward Criterion and Risk
The average reward criterion is used to solve infinite-horizon MDPs. This risk-neutral criterion depends on the stochastic process in the limit and can use (i) the accumulated reward at infinity, which considers sequences of states of size h = ∞, or (ii) the steady state distribution of the MDP (i.e., the probability that the system is in each state in the long term), which considers sequences of states of size h = 1. In many situations, it is desirable to consider risk during the process at each stage, which can be achieved with the average reward criterion using a utility function or a risk measure such as VaR and CVaR. The objective of this work is to propose a mathematical framework that allows a unified treatment of the existing literature using average reward and risk, including works that use exponential utility functions and CVaR, as well as to include interpretations with 1 ≤ h ≤ ∞ not present in the literature. These new interpretations allow differentiating policies that may not be distinguished from existing criteria. A numerical example shows the behaviors of the criteria considering this new framework.
Adaptive Client-Dropping in Federated Learning: Preserving Data Integrity in Medical Domains
In this work, we address the challenge of training machine learning models on sensitive clinical data while ensuring data privacy and robustness against data corruption. Our primary contribution is an approach that integrates Conformal Prediction (CP) techniques into Federated Learning (FL) to enhance the detection and exclusion of corrupted data contributors. By implementing a client-dropping strategy based on an adaptive threshold informed by the interval width metric, we dynamically identify and exclude unreliable clients. This approach, tested using the MedMNIST dataset with a ResNet50 architecture, effectively isolates and discards corrupted inputs, maintaining the integrity and performance of the learning model. Our findings demonstrate that this strategy prevents the potential 10% decrease in accuracy that can occur without such measures, confirming the efficacy of our CP-enhanced FL methodology in ensuring robust and private data handling in sensitive domains like healthcare.
An Ensemble of LLMs Finetuned with LoRA for NER in Portuguese Legal Documents
Given the high computational costs of traditional fine-tuning methods and the goal of improving performance,this study investigate the application of low-rank adaptation (LoRA) for fine-tuning BERT models to Portuguese Legal Named Entity Recognition (NER) and the integration of Large Language Models (LLMs) in an ensemble setup. Focusing on the underrepresented Portuguese language, we aim to examine the reliability of extractions enabled by LoRA models and glean actionable insights from the results of both LoRA and LLMs operating in ensembles. Achieving F1-scores of 88.49% for the LeNER-Br corpus and 81.00% for the UlyssesNER-Br corpus, LoRA models demonstrated competitive performance, approaching state-of-the-art standards. Our research demonstrates that incorporating class definitions and counting votes per class substantially improves LLM ensemble results. Overall, this contribution advances the frontiers of AI-powered legal text mining, proposing small models and initial prompt engineering to low-resource conditions that are scalable for broader representation.
An Instance Level Analysis of Classification Difficulty for Unlabeled Data
Instance hardness measures allow us to assess and understand why some observations from a dataset are difficult to classify. With this information, one may curate and cleanse the training dataset for improved data quality. However, these measures require data to be labeled. This limits their usage in the deployment stage when data is unlabeled. This paper investigates whether it is possible to identify observations that will be hard to classify despite their label. For such, two approaches are tested. The first adapts known instance hardness measures to the unlabeled scenario. The second learns regression meta-models to estimate the instance hardness of new data observations. In experiments, both approaches were better at identifying instances lying in borderline regions of the dataset, which pose a greater difficulty when the label is unknown.
Analyzing the Impact of Coarsening on k-Partite Network Classification
The ever-expanding volume of data presents considerable challenges in storing and processing semi-supervised models, hindering their practical implementation. Researchers have explored reducing network versions as a potential solution. Real-world networks often comprise diverse vertex and edge types, leading to the adoption of k-partite network representation. However, existing methods have mainly focused on reducing uni-partite networks with a single vertex type and edges. This study introduces a novel coarsening method designed explicitly for k-partite networks, aiming to preserve classification performance while addressing storage and processing issues. We conducted empirical analyses on synthetically generated networks to evaluate their effectiveness. The results demonstrate the potential of coarsening techniques in overcoming storage and processing challenges posed by large networks. The proposed coarsening algorithm significantly improved storage efficiency and classification runtime, even with moderate reductions in the number of vertices. This led to over one-third savings in storage space and a twofold increase in classification speed. Moreover, the classification performance metrics exhibited low variation on average, indicating the algorithm’s robustness and reliability in various scenarios.
Applying Transformers for Anomaly Detection in Bus Trajectories
Trajectory anomaly detection is essential in understanding traffic behavior, especially when external factors such as congestion, accidents, and poor weather conditions occur. Most of the previous approaches for this task highly rely on handcrafted features and physical trajectory characteristics, which can be costly to calculate in a large volume of data. In this paper, we propose a novel trajectory anomaly detection approach that relies on language modeling to learn well-formed GPS bus trajectories and, based on it, identifies anomalous trajectories and pinpoints their abnormal points (sub-trajectory anomaly detection). Our solution uses a deep generative encoder-decoder Transformer that learns relationships between the sequential points in the trajectories based on the self-attention mechanism. It does not require manual feature extraction and can be easily adapted to any type of trajectory (e.g., cars, people, and vessels). We have performed an extensive experimental evaluation that shows: (1) our approach is effective for both trajectory and sub-trajectory anomaly detection; and (2) it outperforms the baselines in most evaluation scenarios.
Aroeira: A Curated Corpus for the Portuguese Language with a Large Number of Tokens
The emphasis on constructing extensive datasets for training large language models (LLM) has recently increased, and current literature predominantly features datasets for high-resource languages such as English and Chinese. However, there is a notable scarcity of high-quality corpora for the Portuguese language. To address this limitation, we propose Aroeira, a curated corpus explicitly designed for training large language models in the Portuguese language, with a focus on the Brazilian Portuguese one. The Aroeira Corpus consists of 100 GB of texts from various internet platforms, processed through a comprehensive pipeline to ensure superior quality. The pipeline handles downloading, text extraction, language identification, application of quality and bias filters, and storage, all tailored for the Portuguese language. The resulting corpus contains 35.3 million documents and over 15.1 billion tokens, surpassing the largest previously available corpus in this domain.
Assessing Adversarial Effects of Noise in Missing Data Imputation
In real-world scenarios, a wide variety of datasets contain inconsistencies. One example of such inconsistency is missing data (MD), which refers to the absence of information in one or more variables. Missing imputation strategies emerged as a possible solution for addressing this problem, which can replace the missing values based on mean, median, or Machine Learning (ML) techniques. The performance of such strategies depends on multiple factors. One factor that influences the missing value imputation (MVI) methods is the presence of noisy instances, described as anything that obscures the relationship between the features of an instance and its class, having an adversarial effect. However, the interaction between MD and noisy instances has received little attention in the literature. This work fills this gap by investigating missing and noisy data interplay. Our experimental setup begins with generating missingness under the Missing Not at Random (MNAR) mechanism in a multivariate scenario and performing imputation using seven state-of-the-art MVI methods. Our methodology involves applying a noise filter before performing the imputation task and evaluating the quality of the imputation directly. Additionally, we measure the classification performance with the new estimates. This approach is applied to both synthetic data and 11 real-world datasets. The effects of noise filtering before imputation are evaluated. The results show that noise preprocessing before the imputation task improves the imputation quality and the classification performance for imputed datasets.
Assessing European and Brazilian Portuguese LLMs for NER in Specialised Domains
This paper discusses the impact of Portuguese variants in Large Language Models for the task of named entity recognition (NER) in specialised domains. The tests were made on a Brazilian Portuguese legal and a European Portuguese historical corpora. The models taken into account are BERTimbau (PT-BR), Albertina (PT-PT and PT-BR), and XML-R (multilingual). The impact was more evident in the Portuguese historical corpus, which resulted in higher F1 measures compared to previous works that did not consider the same language variant. Additionally, the study underscores the impact of model architecture on performance, highlighting the critical role of both linguistic alignment and model size in enhancing NER in specialised domains.
BASWE: Balanced Accuracy-Based Sliding Window Ensemble for Classification in Imbalanced Data Streams with Concept Drift
In the wake of the exponential growth in data generation witnessed in recent decades, the binary classification task within data streams presents inherent challenges due to their continuous, real-time flow and dynamic nature. This paper introduces the Balanced Accuracy-based Sliding Window Ensemble (BASWE) algorithm that leverages Balanced Accuracy, sliding windows, and resampling techniques to effectively handle imbalanced classes and concept drifts, ensuring robust performance even as data patterns evolve. In experiments conducted on 40 datasets, comprising 16 real-world and 24 synthetic datasets generated under three configurations-no drift, gradual drift, and sudden drift-and with varying imbalance ratios, BASWE demonstrated superior performance compared to seven other state-of-the-art algorithms in terms of F1 Score and the Kappa statistic.
Beyond Audio Signals: Generative Model-Based Speaker Diarization in Portuguese
Speaker diarization, the task of automatically identifying different speakers in audio and video, is frequently performed using probabilistic models and deep learning techniques. However, existing methods usually rely on direct analysis of the audio signal, which presents challenges for languages that lack established diarization methodologies, such as Portuguese. In this article, we propose a new approach to speaker diarization that leverages generative models for automatic speaker identification in Portuguese. We employed two generative models: one for refining the transcribed audio and another for performing the diarization task, as well as a model for initially transcribing the audio. Our method simplifies the diarization process by capturing and analyzing speaker style patterns from transcribed audio and achieves high accuracy without depending on direct signal analysis. This approach not only increases the effectiveness of speaker identification but also extends the usefulness of generative models to new domains. It opens a new perspective for diarization research, especially for the development of accurate systems for under-researched languages in audio and video applications.
Classification of Non-alcoholic Fatty Liver Disease in Thermal Images of the Liver Using a Siamese Neural Network
Non-alcoholic fatty liver disease (NAFLD) is a prevalent and severe condition that requires effective and non-invasive diagnostic methods. This work presents an innovative approach using Siamese neural networks to classify thermal images of the liver in order to identify the presence of NAFLD. The research is motivated by the need to use artificial intelligence to analyze thermal images, especially when the number of images is limited, making it difficult for ordinary neural networks to learn, a difficulty that the Siamese network already faces with ease. The proposed method involves three stages: extraction of the region of interest (ROI), image pre-processing and classification using the Siamese Neural Network, which compares pairs of images to determine their similarity and, consequently, the presence or absence of NAFLD. The preliminary results, with an accuracy of 71%, precision of 57% and recall of 96%, indicate that this approach could offer a promising tool for the non-invasive diagnosis of NAFLD, contributing a promising method to the field of machine learning applied to medicine.
Classifying Graphs of Elementary Mathematical Functions Using Convolutional Neural Networks
The classification of images of elementary mathematical function graphs presents a significant challenge in computer vision; this is due to the varied shapes and formats of each functions curves. This classification is crucial for identifying function graphs, which have important applications in text and mathematical symbol recognition technologies, aiding visually impaired individuals by providing access to printed content. In educational environments, this identification helps obtain the analytical expression of drawn graphs, facilitating the extraction of information from educational materials. This article investigates various convolutional neural network (CNN) architectures to identify the most suitable model for classifying images of elementary mathematical function graphs. We compare our model with other renowned architectures, such as ResNet, MobileNet, and EfficientNet, using a custom dataset of function graphs. Our experiments show that the proposed architecture significantly outperforms networks of general purpose, achieving an accuracy of 98.51% in classifying elementary mathematical function graphs.
Comparing Neural Network Encodings for Logic-Based Explainability
Providing explanations for the outputs of artificial neural networks (ANNs) is crucial in many contexts, such as critical systems, data protection laws and handling adversarial examples. Logic-based methods can offer explanations with correctness guarantees, but face scalability challenges. Due to these issues, it is necessary to compare different encodings of ANNs into logical constraints, which are used in logic-based explainability. This work compares two encodings of ANNs: one has been used in the literature to provide explanations, while the other will be adapted for our context of explainability. Additionally, the second encoding uses fewer variables and constraints, thus, potentially enhancing efficiency. Experiments showed similar running times for computing explanations, but the adapted encoding performed up to 18% better in building logical constraints and up to 16% better in overall time.
Deep Learning Approach to Temporal Dimensionality Reduction of Volumetric Computed Tomography
A common approach for analyzing medical images on volumetric data is to employ deep 2D convolutional neural networks (2D CNN) on each individual slice of the volume. This is largely attributed to the challenges posed by the nature of three-dimensional data: variable volume size, high GPU and RAM requirements, costly parameter optimization, etc. However, handling the individual slices independently in 2D CNNs deliberately discards the temporal information that is contained within the depth of the volumes, which tends to result in poor performance. In order to maintain temporal information, current solutions reduce the temporal dimensionality of the data using non-adaptive sampling processes, such as taking slices at given intervals or by means of interpolation. However, although this allows to keep some temporal information, it may discard important slices. In this paper, we propose a method based on GradCam to select meaningful slices in computed tomography volumes by evaluating the activation map to reduce temporal data dimensionality. Extensive experiments demonstrate the effectiveness of our method when compared to the current state of the art.
Deployment of IBM Federated Learning Platform and Aggregation Algorithm Comparison: A Case Study Using the MNIST Dataset
With the exponential growth in artificial intelligence, privacy and data acquisition concerns have surged, prompting stricter data protection laws. Federated Learning (FL) addresses these issues by enabling model training without accessing private data, allowing geographically dispersed clients to participate without sharing data. This study deployed an IBM FL platform using Docker containers across two clients and conducted training with the MNIST dataset using the two most common FL strategies: Federated Stochastic Gradient Descent (FedSGD) and Federated Averaging (FedAvg). The results validated the platform’s deployment and assessed the performance of each strategy in terms of model accuracy and client hardware capabilities. Performance metrics, including CPU and RAM usage, network traffic, and model accuracy, were collected. Despite the higher resource demands, both strategies achieved satisfactory model accuracy, with FedAvg showing slightly better efficiency for the small-scale deployment. The results emphasize the potential of FL for predictive maintenance in industrial applications, enabling decentralized data utilization while ensuring data privacy and security.
Detection of Pathological Regions of the Gastrointestinal Tract in Capsule Images Using EfficientNetV2 and YOLOv8
Diseases of the gastrointestinal (GI) tract are among the most common pathologies in the world population and are responsible for thousands of deaths every year. This work proposes an automatic method for detecting regions with GI Tract abnormalities, intending to reduce the number of lesions missed in Wireless Capsule Endoscopy (WCE) video exams by expert endoscopists. By taking advantage of convolutional neural networks (CNNs) and YOLO detection models, the proposed method not only increases the reliability of pathological detection in WCE images, but also sets a new benchmark in this field. Our results for binary classification between healthy and pathological images are promising, with an accuracy of 87.8%, precision of 91.6%, recall of 89.1% and F1-Score of 90.3%. In addition, the detection model showed an Intersection over Union (IoU) of 31.33% among all the images classified as pathological. The impact of this research is significant, as it provides a method capable of detecting GI Tract diseases in WCE images and contributing to better clinical decision-making and patient care.
Dual-Bandwidth Spectrogram Analysis for Speaker Verification
The variability of the human voice is a challenge for speaker verification systems, influenced by individual traits and environmental conditions. This research introduces a novel approach that uses dual-bandwidth spectrograms with the Fast ResNet-34 neural network architecture for speaker verification. Dual-bandwidth spectrograms are data structures similar to multi-channel images, generated by stacking spectrograms derived from the same audio segment using two different window sizes. In this study, we employed window sizes of 5 ms and 30 ms. This approach captures a wider range of voice features across multiple temporal and spectral resolutions. Our findings demonstrate a statistically significant improvement in system performance, achieving an Equal Error Rate (EER) of 1.64% ±0.13%. This represents a 26% enhancement over the previously reported benchmark EER of 2.22% ±0.05%, validating our hypothesis that dual-bandwidth spectrograms offer a more detailed and comprehensive representation of voice features for accurate speaker verification. Analysis of individual bandwidth contributions reveals that narrowband spectrograms carry more relevant features for speaker verification, while the combination with broadband spectrograms provides complementary information.
Dynamicity Analysis in the Selection of Classifier Ensembles Parameters
Over the years, significant progress has been made in the realm of classifier ensembles research. Several methods to enhance their efficiency have been proposed, applicable to both homogeneous and heterogeneous ensemble structures. A key challenge in employing classifier ensembles lies in determining their structure (hyper-parameters). Basically, the ensemble structure selection can be done in two different ways, static and dynamic selection. Unlike static selection, dynamic selection defines the ensemble structure for each testing instance. Different dynamic selection methods have been proposed in the literature, mainly for ensemble members and features, but very little effort has been done to propose dynamic selection methods for combination methods. Therefore, it is important to evaluate the impact of a dynamic selection of combination methods or both (methods and members) in the creation of robust classifier ensembles. In this paper, an exploratory analysis of dynamic selection of the main ensemble structure parameters will be performed. In order to do this, three different scenarios will be assessed: Full Static ensemble, Partial Dynamic ensemble and Full Dynamic ensemble. Finally, an empirical analysis of these scenarios will be carried out. Our findings show that the use of a full dynamic selection provides more robust classifier ensembles, in most of the analyzed cases.
Embedding Representations for AutoML Pipelines
The area of Automated Machine Learning (AutoML) emerged to automate the tedious process of manually testing different sets of algorithms hyperparameters and other data engineering tasks involved in the process of solving a machine learning (ML) problem. While many researchers focus on developing and refining techniques, there have been few advances in understanding the models and how the optimization works. One way to tackle this problem is to investigate the fitness landscape and analyze the distance between solutions to describe this environment. These techniques require calculating distances between solutions in the search space. This is a problem, as in a diverse range of AutoML methods, machine learning pipelines are represented by a tree structure, which has limitations in computational time for calculating distances and does not account for the semantics for the solutions. In this direction, this paper proposes a new way to represent ML pipelines using embeddings. We use a Transformer model to generate embeddings of machine learning pipelines, and then evaluated the embeddings using the correlation between the distances calculated when using the two representations. We also perform a qualitative and a visual analyses to compare both representations. Developing this representation allows researchers to improve current AutoML methods by providing a better understanding of how difficult it is to search for them.
Enhancing Graph Data Quality by Leveraging Heterogeneous Node Features and Embeddings
Heterogeneous Graphs are important data sources due to their rich representation of knowledge, primarily based on node features and relationships. It is common for these graphs to have significant data gaps, particularly in the nodes. Graph Neural Networks are state-of-the-art solutions that achieve excellent results by extracting information based on node relationships. However, they suffer from severe limitations when there is no available information in the graph elements, weakening their representation. This paper proposes the specifications and an algorithm to process different types of node features, such as text, images, and subgraphs, generating both single and composition embeddings. To evaluate the effectiveness of the proposed algorithm, experiments were conducted to generate the features and their respective node embeddings in a Heterogeneous Graph. The achieved performance was measured using the average of Accuracy, F1-Score, and their Standard Deviations based on the Recommender System tasks applied to the embeddings generated in the experiments. We can highlight the performance achievement in the experiments as the Node Classification task, using the composition of Aggregated Features with Metapaths embedding, which achieved an F1-Score of 83.66% overcoming the 60.70% achieved by the approach without embeddings.
Ensemble of CNNs for Enhanced Leukocyte Classification in Acute Myeloid Leukemia Diagnosis
Acute Myeloid Leukemia (AML) is one of the most lethal and aggressive forms of hematological cancer, characterized by the rapid proliferation of immature leukocytes. This disease is diagnosed by highly trained specialists who meticulously analyze microscopic images of blood smears. This work explores the feasibility and effectiveness of using eight Convolutional Neural Network (CNN) architectures to form specialized ensembles capable of accurately differentiating between mature and immature leukocytes. We used voting ensemble techniques and the bagging method to integrate CNNs that achieved the best individual performances. The bagging strategy, explicitly using the EfficientNet B3 CNN, stood out by achieving an accuracy of 96.62%, a precision of 98.11%, and a Kappa index of 92.27% on a dataset of 48,100 blood cell images. This performance enhancement highlights the superior diagnostic capabilities of this approach compared to the individual architectures of CNNs in identifying cell types in the context of AML diagnosis.
ERASMO: Leveraging Large Language Models for Enhanced Clustering Segmentation
Cluster analysis plays a crucial role in various domains and applications, such as customer segmentation in marketing. These contexts often involve multimodal data, including both tabular and textual datasets, making it challenging to represent hidden patterns for obtaining meaningful clusters. This study introduces ERASMO, a framework designed to fine-tune a pretrained language model on textually encoded tabular data and generate embeddings from the fine-tuned model. ERASMO employs a textual converter to transform tabular data into a textual format, enabling the language model to process and understand the data more effectively. Additionally, ERASMO produces contextually rich and structurally representative embeddings through techniques such as random feature sequence shuffling and number verbalization. Extensive experimental evaluations were conducted using multiple datasets and baseline approaches. Our results demonstrate that ERASMO fully leverages the specific context of each tabular dataset, leading to more precise and nuanced embeddings for accurate clustering. This approach enhances clustering performance by capturing complex relationship patterns within diverse tabular data.
Euclidean Alignment for Transfer Learning in Multi-band Common Spatial Pattern
Stroke is a major cause of death and disability worldwide. Motor-Imagery based Brain-Computer Interface (MI-BCI) models offer a post-stroke rehabilitation option. Existing studies for MI-BCI use Transfer Learning techniques like Euclidean Alignment (EA) but lose important brain information due to bandpass filtering. This study introduces new BCI architecture with multi-band temporal filters and EA. The methods considered here are Filter Bank (FB), Empirical Mode Decomposition (EMD), and Continuous Wavelet Transform (CWT). Results show performance improvements, especially with EA being applied before Filter Bank. These models offer promise for post-stroke rehabilitation, particularly when using EA before the multi-band filter.
Evaluating CNN-Based Classification Models Combined with the Smoothed Pseudo Wigner-Ville Distribution to Identify Low Probability of Interception Radar Signals
Fundamental in defense, Radar Electronic Warfare (REW) requires adaptation to current threats. Automatic recognition algorithms for intrapulse modulations (ATR) of Low Probability of Interception (LPI) radar signals are essential in REW. Existing LPI signal ATR methods combine the Choi-Williams Distribution (CWD) pre-processing technique with Convolutional Neural Networks (CNN). This work proposes two new ATR combinations, based on SqueezeNet and GoogLeNet CNN. Both used the Smoothed Pseudo-Wigner-Ville distribution (SPWVD) pre-processing technique as an alternative to CWD. Replacing CWD by SPWVD was based on the hypothesis that the latter usually provides higher resolutions than the former. The proposed ATR overcame the SOTA ATR, achieving a 99.06% accuracy, under noisy environments and providing evidence to the hypothesis raised. Experiments involved two datasets with 13 types of modulations and 806,000 samples each.
Evaluating Large Language Models for Tax Law Reasoning
The ability to reason over laws is essential for legal professionals, facilitating interpreting and applying legal principles to complex real-world situations. Tax laws are crucial for funding government functions and shaping economic behavior, yet their interpretation poses challenges due to their complexity, constant evolution, and susceptibility to differing viewpoints. Large Language Models (LLMs) show considerable potential in supporting this reasoning process by processing extensive legal texts and generating relevant information. This study evaluates the performance of LLMs in legal reasoning within the domain of tax law for legal entities, utilizing a dataset of real-world questions and expert answers in Brazilian Portuguese. We employed quantitative metrics (BLEU, ROUGE) and qualitative assessment using a solid LLM to ensure factual accuracy and relevance. A novel dataset was curated, comprising genuine questions from legal entities in tax law, answered by legal experts with corresponding legal texts. The evaluation includes both open-source and proprietary LLMs, providing a assessment of their effectiveness in legal reasoning tasks. The strong correlation between robust LLM evaluator metric and Bert Score F1 suggests these metrics effectively capture semantic aspects pertinent to human-perceived quality.
Explaining Biomarker Response to Anticoagulant Therapy in Atrial Fibrillation: A Study of Warfarin and Rivaroxaban with Machine Learning Models
Atrial fibrillation (AF) is a common arrhythmia that originates in the heart’s upper chambers and can lead to serious complications like strokes and systemic embolism due to atrial thrombi. To mitigate these risks, anticoagulants such as warfarin and rivaroxaban are frequently prescribed. This study examines how known biomarkers behave under treatment with either warfarin or rivaroxaban. We developed high-performance models (AUROC > 0.94) to distinguish patient subpopulations based on their treatment. Additionally, we built models (AUROC > 0.97) to differentiate individuals with AF from a control group without the condition. Using synthetic data generation for training augmentation and explainable machine learning techniques, we analyzed biomarker behavior, uncovering distinct patterns based on whether patients received warfarin or rivaroxaban. Our approach provides valuable insights into the critical factors influencing biomarker variations across different treatments, enhancing our understanding of their roles in AF management.
Going Bananas! - Unfolding Program Synthesis with Origami
Automatically creating a computer program using input-output examples can be a challenging task, especially when trying to synthesize computer programs that require loops or recursion. Even though the use of recursion can make the algorithmic description more succinct and declarative, this concept creates additional barriers to program synthesis algorithms such as the creation and the attempt to evaluate non-terminating programs. One reason is that the recursive function must define how to traverse (or generate) the data structure and, at the same time, how to process it. In functional programming, the concept of Recursion Schemes decouples these two tasks by putting a major focus on the data processing. This can also help to avoid some of the pitfalls of recursive functions during program synthesis, as argued in the paper that introduced the Origami technique. The authors showed how this technique can be effective in finding solutions for programs that require folding a list. In this work, we incorporate other Recursion Schemes into Origami, such as accumulated folding, unfolding, and the combination of unfolding and folding. We evaluated it on the 29 problems of the standard General Program Synthesis Benchmark Suite 1, obtaining favorable results against other well-known algorithms. Overall, Origami achieves the best results in more problems than its predecessor (HOTGP) and an even higher increase when compared to other approaches. Not only that, but it can also consistently find a solution to problems for which other concurrent algorithms report a low success rate.
GovBERT-BR: A BERT-Based Language Model for Brazilian Portuguese Governmental Data
Given the growing interest in natural language processing (NLP) for governmental applications, particularly in Brazil, where vast amounts of governmental data are processed daily, the need for specialized NLP models tailored to the nuances of Brazilian Portuguese and the legal and administrative domains has become increasingly apparent. However, existing models may struggle to accurately interpret the complexities of governmental texts, often leading to suboptimal performance in document classification and analysis tasks. To address these challenges, we introduce GovBERT-BR, a pre-trained language model tailored to the Brazilian governmental context, covering legal and administrative domains. Leveraging insights from diverse governmental texts, GovBERT-BR addresses the challenges of accurately interpreting Brazilian Portuguese and the unique legal and bureaucratic terminology prevalent in governmental documents. We present the pre-training process and experimental evaluation of GovBERT-BR, comparing its performance against baseline models across various text classification tasks relevant to the Brazilian public sector. Our findings demonstrate that GovBERT-BR outperforms existing models in document and short-text classification tasks, showcasing its efficacy in accurately analyzing governmental text data. Furthermore, our analysis reveals insights into the convergence behavior of GovBERT-BR during fine-tuning, highlighting its rapid adaptation to downstream tasks.
Growing Self-Organizing Maps for Multi-label Classification
In Machine Learning, multi-label classification is the problem of simultaneously classifying an instance into two or more labels. It is a challenging problem since each label has its specialty, and correlations between them must be considered. A Self-Organizing Map (SOM) is a Neural Network where neurons organized in a grid are tuned to represent the input instances in self-organization. After tuning, similar instances in the input space are mapped to closer neurons in the grid. SOMs have already been used for multi-label problems, obtaining competitive results with other methods. However, the static nature of their grid of neurons is a disadvantage since it is difficult to define the optimized grid size for each problem. The Growing Self-Organizing Maps (GSOM) extends the SOMs, allowing the network to grow during execution based on the data characteristics. This paper proposes a GSOM to predict multi-label data. The experiments showed that GSOM obtained better or more competitive results in most of the datasets investigated compared to SOM and had a competitive performance compared to other methods.
HEACT: Hybrid Evolutionary Algorithm for the Multi-region Multi-objective Cloud Task Scheduling Problem. A Study of Workflow Scheduling in AWS EC2
Cloud computing has revolutionized the provisioning and access of computing resources, offering scalable and flexible alternatives to traditional infrastructure. However, defining how to use these computational resources may be challenging. This paper addresses the challenge of workflow scheduling in cloud environments, focusing on Amazon Web Services (AWS) Elastic Compute Cloud (EC2). We present HEACT, a novel approach that integrates a multi-objective evolutionary algorithm with a specialist scheduling heuristic. The evolutionary algorithm is responsible for generating an initial set of machines (with their performance capability and cost information). The set is sent to the specialist scheduling heuristic for efficient task assignment in these machines. Our approach considers fourteen AWS regions, accurate pricing information from AWS, and employs SimGrid to simulate task execution. The proposed method was benchmarked considering established heuristics (HEFT, PEFT, HSIP, MPEFT) and meta-heuristics (NSGA-II, AGEMOEA2). Results demonstrated that the combinations of AGEMOEA2 with MPEFT and AGEMOEA2 with HEFT yield the best performance, indicating AGEMOEA2’s efficacy as a state-of-the-art meta-heuristic for workflow scheduling.
Heuristic Solutions for the 2D Bin-Packing Problem with Varied Size
This work addresses the 2D Bin-Packing Problem with Varied Size and proposes heuristic solutions for it. An extensive literature review on the problem is carried out, and the state-of-the-art literature algorithm for its variation with guillotine constraint is reproduced in order to validate the reported results. It is also used to build a baseline which is compared with a proposed improvement for this algorithm in order to enable the addition of a new constraint to the problem that meets an industry demand for its practical application.
Humanities and AI: Ethical Education in Technology Careers
This paper presents a quantitative and qualitative study of the teaching of ethics and related subjects in undergraduate courses in Computing of 17 top-ranked universities evaluated in three different ranking systems. The analysis was performed using documents from the selected universities and semi-structured interviews with professors and a head of department. This article discusses the relevance of ethical and social-oriented teaching and classifies the syllabuses of the sampled disciplines, assessing the type of focus that is apparently given. The overall results show considerable variability in the availability of data, ranging from courses whose content is very detailed to those that don’t present their syllabus at all. Considering that the development of critical reasoning based on ethics is extremely relevant to the education of future Artificial Intelligence developers, the authors concluded that universities are not fulfilling their role adequately.
Impact of Parent Selection Operator on the FDEA Algorithm
Many-Objective Optimization Problems (MaOPs) present a significant challenge due to the increased complexity associated with optimizing more than three conflicting objectives. Traditional Pareto-based algorithms often struggle to scale effectively with the number of objectives, leading researchers to explore alternative approaches. This study investigates the impact of parent selection strategies on the performance of the Fuzzy Decomposition-Based Multi/Many-Objective Evolutionary Algorithm (FDEA). The FDEA uses a fuzzy decomposition approach to divide a MaOP into subproblems, each solved individually to improve the adaptability and precision of solutions. The study compares the FDEA with three selection methods: binary tournament based on Pareto dominance, binary tournament using crowding distance, and a combination of Pareto dominance and crowding distance. Empirical evaluations were conducted using the DTLZ and WFG benchmark problem sets, with 2, 3, 5, 8, 10, and 15 objectives. The results indicate that incorporating quality indicators into the selection process improves the efficiency and effectiveness of the FDEA, especially in high-dimensional optimization scenarios. However, no variant was superior in all contexts. The study highlights the potential for methodological advances in Indicator-Based Mating Selection, emphasizing the need for robust and scalable algorithms capable of handling the complexities of MaOPs.
Improving LLMs’ Reasoning and Planning with Finite-State Machines
Large Language Models (LLMs) have shown remarkable capabilities across various applications but struggle with sequential decision-making tasks like planning. This work demonstrates that integrating LLMs with Finite-State Machines (FSMs) can enhance their reasoning and planning abilities, while also offering increased reliability. Several setup variants are compared, providing a better understanding of how validations, feedback loops, and restrictions enhance robustness and effectiveness. A comparison to the well known Chain-of-Thoughts approach is also provided. Our methods improve planning capabilities of all analysed LLMs, consistently increasing the success rate in solving tasks of varying complexities. A detailed analysis of the two best variants are provided, highlighting their respective strengths and weaknesses.
Improving Short-Content Misinformation Detection Using Multiple Aspect Trajectories Classification Techniques
The proliferation of fake news and misinformation on social media poses a significant threat to social integrity. While there are approaches such as automatic content analysis and artificial intelligence detection to mitigate this problem, these methods face challenges in classifying misinformation in short content, like that posted on microblogging platforms or instant messaging services. In this study, we present an innovative approach to misinformation detection that combines content-based detection with multi-aspect trajectory classification. This approach models information propagation by considering each shared message as a moving object within the network, enriching the propagation trajectory with different semantic aspects. We implemented this approach on a misinformation dataset collected from WhatsApp and compared it with traditional content-based detection methods, and we combined these two detection orientations into a hybrid approach. Our findings indicate that the proposed hybrid approach achieves an F1-score of 0.89, surpassing baseline models by 8% in misinformation detection, even in short content. These results suggest that combining content-based detection with multi-aspect trajectory classification is a novel and promising strategy for addressing misinformation on social media.
InRanker: Distilled Rankers for Zero-Shot Information Retrieval
Despite multi-billion parameter neural rankers being common components of state-of-the-art information retrieval pipelines, they are rarely used in production due to the enormous amount of compute required for inference. In this work, we propose a method for distilling large rankers into their smaller versions focusing on out-of-domain effectiveness. We introduce InRanker, a version of monoT5 [25] distilled from monoT5-3B with increased effectiveness on out-of-domain scenarios. Our key insight is to use language models and rerankers to generate as much as possible synthetic “in-domain” training data, i.e., data that closely resembles the data that will be seen at retrieval time. The pipeline consists of two distillation phases that do not require additional user queries or manual annotations: (1) training on existing supervised soft teacher labels, and (2) training on teacher soft labels for synthetic queries generated using a large language model. Consequently, models like monoT5-60M and monoT5-220M improved their effectiveness by using the teacher’s knowledge, despite being 50x and 13x smaller, respectively. Furthermore, we show that it is possible to transfer knowledge from English models to Portuguese fine-tuned models. Models and code are available at https://github.com/unicamp-dl/inranker.
Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving Based on Bird’s-Eye View in Simulated Cities
This paper investigates the use of Behavior Cloning (BC) for autonomous driving from a bird’s-eye view (BEV) perspective in simulated urban environments. BC uses supervised learning to mimic expert driving behaviors. Previous works have applied BC in the CARLA simulator but did not fully address the challenges of traffic light compliance. Our approach enhances BC by integrating a kernel density estimator to adjust training sample weights based on action density, thereby improving the learning of rare but critical actions such as stopping at red lights and accelerating at green lights, specially in scenarios of scarce number of expert demonstrations. Using BEV inputs, which provide an abstract top-down view of the driving environment, our method simplifies the policy learning process. The trained convolutional neural network (CNN) outputs steering and acceleration actions based on these BEV inputs and additional state variables. Experimental results in the CARLA simulator demonstrate that our weighted BC method significantly improves driving performance, achieving higher route completion compared to standard BC. This weighted approach proved to be crucial in learning correct driving behaviors, particularly in test environments not encountered during training, highlighting its potential for enhancing autonomous vehicle navigation.
Investigating Universal Adversarial Attacks Against Transformers-Based Automatic Essay Scoring Systems
Automatic Essay Scoring promises to scale up student feedback on written input, addressing the excessive cost and time demand associated with human grading. State-of-the-art automatic scorers are based on Transformers-based neural networks. While such models have shown impressive results in reasoning tasks, learned models often produce answers that arise from statistical clues in datasets and are misaligned with human objectives. Such systems are thus potentially fragile for scenarios where users are incentivized to deceive the system, as in a classroom setting. In this work, we evaluate the susceptibility of state-of-the-art automatic scorers to attacks made by non-expert users, such as students interacting with an automatic grader. We develop a methodology to simulate such student attacks and test them against scorers based on BERT, Phi-3 and Gemini models. Our findings suggest that (i) a BERT-based grader can be deceived using simple feature-based attacks; (ii) although Google’s Gemini has a solid agreement with graders, it can assign undeservedly high grades for small sentences; (iii) a Phi-3-based grader was less susceptible than BERT, but it still assigned relatively high grades to some of our attacks.
Likelihood Estimator for Multi Model-Based Reinforcement Learning
Efficiently learning and generalizing from interactions with the environment is a concern in reinforcement learning. Model-based approaches offer promising potential to address these challenges, leveraging the underlying structure of the world to speed learning and improve generalization. By assuming a structured environment, such approaches aim to capture key patterns from limited interactions and synthesize experiences for enhanced exploitation and generalization to unexplored territories. In this paper, we test a model-based reinforcement learning framework aimed at further advancing the frontiers of efficiency and generalization. Motivated by the insight that environments often exhibit distinct regions with varying dynamics, we introduce additional assumptions about the structure of the world to facilitate faster generalization. Specifically, we construct a model comprising a classifier that, given a state input, selects parameters characterizing the operating regime of the environment. Both the model and the regional parameters are learned from past experiences using a likelihood function. Leveraging this model, we employ a rapid exploring random tree based planner to generate new real experiences, capitalizing on the identified structural nuances within the environment. To evaluate the efficacy learning a segmented model, we conduct a comparative analysis against a traditional method that employs a single general model to learn the dynamics of the entire environment. Our results demonstrate the superiority of the segmented approach in terms of both efficiency and generalization, underscoring the benefits of incorporating additional assumptions about the structure of the world into model-based reinforcement learning paradigms.
LLM-Driven Chest X-Ray Report Generation With a Modular, Reduced-Size Architecture
Large Language Models (LLMs) have been widely employed in various text processing tasks. In computer vision, these models have found application in generating captions and text from natural images, as well as in Visual Question Answering (VQA) systems. In the field of medical imaging, there are studies based on text generation proposing automated diagnoses of X-rays, magnetic resonance imaging scans, computed tomography scans, and other modalities. Few initiatives seek to apply and harness the potential of LLMs in medical text generation; they use models with tens of billions of parameters and are thus computationally expensive. This work addresses this gap by evaluating the use of frozen pre-trained models (CXAS U-Net and BioGPT) for chest X-ray report generation. We adapt the BLIP-2 modular architecture where only a cross-modal alignment module must be trained in order to generate text from images. We were able to achieve competitive scores over Clinical Efficacy (CE) metrics compared to some state-of-the-art (SOTA) methods, while obtaining lower scores for Natural Language Generation (NLG) metrics. Our findings suggest that NLG metrics may not serve as suitable proxies for evaluating models in the chest X-ray generation task.
Multilingual Extractive Summarization: Investigating State-of-the-Art Methods for English and Brazilian Portuguese
Automatic Text Summarization (ATS) is a Natural Language Processing (NLP) task essential for handling large volumes of information. ATS can be classified into two main types: extractive and abstractive. Extractive summarization selects sentences or phrases directly from the source text(s), while abstractive summarization generates new sentences that try to capture the original meaning of the source text(s). This paper describes our efforts to perform extractive single-document summarization in multilingual contexts. Although various summarization methods, such as PreSumm and HiStruct+, have shown promising results on English corpora like CNN/DM, there is a significant gap in applying these methods to other languages, especially Brazilian Portuguese. Additionally, these summarizers were evaluated with traditional metrics like ROUGE, which has limitations as it primarily measures superficial text overlap. To fill these gaps, we evaluate the effectiveness of these state-of-the-art methods on the CSTNews corpus (with news texts in Brazilian Portuguese) employing ROUGE and the recent BLANC metric, which measures how much the generated summary aids a pre-trained language model (like BERT) in understanding the document. Our contributions include the results and comparison of adapted models, the discussion of the BLANC metric in contrast to ROUGE, and the expansion of resources available to the Portuguese and multilingual NLP community.
Multimodal and Hybrid Models for Predicting SCD Risk in Chagas Cardiomyopathy
Predicting sudden cardiac death (SCD) in Chagas cardiomyopathy (CC) remains a challenge. An estimated 6 to 7 million people worldwide are affected by this condition. To address this challenge, we propose a novel intelligent system that utilizes multimodal and hybrid machine learning (ML) models to predict SCD risk in CC patients. We evaluated different approaches to combine clinical features (tabular data) and temporal features extracted from ECG-Holter exams. Our results indicate that the hybrid model (RNN with MLP) achieved the highest recall (sensitivity) of 91.63% and 81.08% of AUC. This study aims to develop a predictive model to support clinical decision-making in the context of SCD in CC. The model can potentially aid in risk stratification, enabling early intervention and potentially saving lives.
On the Equivalence Between Logic Programs and Bipolar Argumentation Frameworks
In this paper, we extend the connection between argumentation formalisms and logic programs, by showing the equivalence between bipolar argumentation β-semantics and various 3-valued logic programming semantics. In particular, we show that β-semi-stable semantics corresponds to L-stable semantics, which has been previously shown not to be expressible by traditional attack-only argumentation frameworks. Besides enhancing our understanding of how logic programs relate to argumentation with support, we provide direct translations between them and their semantics, allowing their interchangeable use.
Optimizing CleanUNet Architecture Parameters for Enhancing Speech Denoising
Speech enhancement refers to a set of techniques aiming to recover clean speech from a corrupted signal. One way to corrupt a signal is through noise addition. Noise comes in a variety of ways. Suboptimal acoustic conditions can cause background noise and echo, hampering speech clarity and making denoising techniques necessary to enhance the speech. In this work, we optimized CleanUNet, a convolutional neural network (CNN) architecture proposed specifically for causal speech denoising tasks. We explored alternatives for the transformer bottleneck, such as Mamba architecture, capable of handling encoder outputs more efficiently with linear complexity, we also reduced the number of hidden channels in the convolutional layers. This decreases the model’s parameter count and improves training and inference speed on a single GPU, offering a streamlined approach for enhanced performance. To our understanding, this is the first attempt to incorporate Mamba as a replacement for the vanilla transformer in the CleanUnet architecture.
Portuguese Emotion Detection Model Using BERTimbau Applied to COVID-19 News and Replies
In this study, we developed an Emotion Detection (ED) model for Brazilian Portuguese social media comments and used it to analyze emotion distribution in replies to COVID-19-related news on Twitter. We translated the GoEmotions dataset into Portuguese using a Large Language Model, preserving the original social media writing style, and fine-tuned the BERTimbau model for detecting Ekman’s six basic emotions plus neutral emotion. We collected tweets related to the COVID-19 pandemic from major Brazilian news portals and analyzed user replies for expressed emotions. Chi-square tests indicated that emotion distribution depended on both the news topic and the media outlet, while the Augmented Dickey-Fuller (ADF) test showed that some emotions’ distributions were influenced by the publication date. Among replies with identifiable emotions, anger (21.1%), joy (17.2%), and surprise (11.6%) were most prevalent. The Brazilian pandemic inquiry (CPI) prompted the highest anger prevalence (41.9%), while news from UOL Notícias had the highest anger (40.0%) and g1 led in sadness (13.6%).
Predicting Bull and Bear Markets: A Deep Learning and Linear Regression Study in Cryptocurrencies
Despite the growing popularity and increasingly widespread use of cryptocurrencies in contemporary financial markets, understanding market trends and predicting their future movements is a formidable challenge in financial analysis. Regardless of the numerous tools and techniques available, accurately forecasting market behavior remains complex due to financial data’s inherent volatility and noise. This work proposes a novel hybrid model for market trend classification and trend slope regression to address this gap. The proposed approach leverages technical indicators, autoencoders, a BiLSTM-CNN architecture, and the use of linear regression to label the trends, enhancing prediction accuracy. The core innovation lies in the integration of these advanced techniques to extract meaningful and useful patterns from the data. This methodology allows the model to better capture the complexities of financial time series, thus improving its predictive capabilities. As a proof of concept, the proposed model was evaluated using historical Bitcoin data spanning two years, with performance metrics such as accuracy, precision, recall for trend classification, and mean squared error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) for trend slope regression. The results are compelling, demonstrating that the combination of linear regression labeling and the architecture proposed achieves a remarkable level of accuracy both in-sample and out-of-sample. This accuracy significantly surpasses the 50% random benchmark in the predictions of bull and bear markets. Ultimately, this research presents a promising approach to enhancing the precision of trend predictions in the volatile and unpredictable cryptocurrency market.
Predicting Energy Consumption Data Using Deep Learning: An LSTM Approach
Nowadays, most of the energy produced globally comes from fossil fuels. However, this type of energy generation harms the environment by emitting various toxic residues in natural bodies. In recent years, an area that has gained strength is producing and consuming clean energy. One that stands out is solar energy production because it is easy to implement and relatively cheap compared to other clean energy production. Therefore, several projects were created focusing on generating energy from sunlight, and using it. Due to this growing number of enterprises in the area, the creation of applications to help manage production and its use has increased. The use of Deep Learning techniques to help this industry has also gained strength; predictive models for energy consumption have been widely studied to help enterprises in future planning. In this work, we developed a deep learning model using Long Short-Term Memory (LSTM) capable of predicting energy consumption from solar power plant data, using the data to train the model and make inferences in the future. We explored configuration combinations such as data filtering with smoothing techniques, model hyperparameters, number of layers, number of neurons, and optimal prediction horizon. The achieved results demonstrate the validity and effectiveness of the implemented methodology.
Pseudonymization in Legal Texts According to the LGPD: A Named Entity Recognition Approach
This study explores the application of Named Entity Recognition (NER) for the pseudonymization of data in legal texts, aiming to protect Personally Identifiable Information (PII) in compliance with Brazil’s General Data Protection Law (LGPD). The research highlights the challenge of balancing data privacy and utility, presenting a methodology that uses artificial intelligence technologies to effectively identify and obscure PII in legal documents. In this study, we propose a Transformer model along with Regex techniques to identify entities in a text. To test the model, we created a new dataset from the existing LenerBR. We also used a function and prompt engineering applied to the Llama 8B version 3 model to generate synthetic data. Tests showed the need for further adjustments in the proposed new model. Future work will focus on improving the model’s accuracy and efficiency, as well as enhancing the identification of sensitive data and learning from user interactions.
ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language
Despite advancements in Natural Language Processing (NLP) and the growing availability of pretrained models, the English language remains the primary focus of model development. Continued pretraining on language-specific corpora provides a practical solution for adapting models to other languages. However, the impact of different pretraining settings on downstream tasks remains underexplored. This work introduces ptt5-v2, investigating the continued pretraining of T5 models for Portuguese. We first develop a baseline set of settings and pretrain models with sizes up to 3B parameters. Finetuning on three Portuguese downstream tasks (ASSIN2 STS, ASSIN2 RTE, and TweetSentBR) yields SOTA results on the latter two. We then explore the effects of different pretraining configurations, including quality filters, optimization strategies, and multi-epoch pretraining. Perhaps surprisingly, their impact remains subtle compared to our baseline. We release ptt5-v2 pretrained checkpoints and finetuned MonoT5 rerankers on HuggingFace in their respective collections at https://huggingface.co/unicamp-dl.
Question Answering with Texts and Tables Through Deep Reinforcement Learning
This paper proposes a novel architecture to generate multi-hop answers to open domain questions that require information from texts and tables, using the Open Table-and-Text Question Answering dataset for validation and training. One of the most common ways to generate answers in this setting is to retrieve information sequentially, where a selected piece of data helps searching for the next piece. As different models can have distinct behaviors when called in this sequential information search, a challenge is how to select models at each step. Our architecture employs reinforcement learning to choose between different state-of-the-art tools sequentially until, in the end, a desired answer is generated. This system achieved an F1-score of 19.03, comparable to iterative systems in the literature.
Reinforcement Learning with Utility-Based Semantic for Goals
Stochastic shortest path problems (SSPs) are Markov decision processes with goal states and the problem is to find policies to achieve the goal with the lowest possible expected cost. Something common in this type of problem is the existence of states from which it is not possible to reach the goal, these states are called dead-end. In this case, it is important to have methods that consider the importance, not only of cost, but also of the probability of achieving the goal. In Reinforcement Learning (RL), the problem of making this trade-off between cost to goal and probability to goal has been little studied and the common strategy to deal with SSPs with dead-ends is using discounts. In some works, penalties are used on exploration in SSPs with dead-ends. However, using discounts and penalties can lead to errors in finding a policy with a desired trade-off. The GUBS criterion (Goals with Utility-Based Semantic) considers this type of trade-off without using discounts or penalties, has good semantics based on the expected utility theory and has been used to solve SSPs with dead-end states in the planning area. Thus, in this work, the GUBS criterion is used to propose the first two algorithms for RL to do a trade-off between probability to goal and cost to goal, without using discounts or penalties: the Q-learning-GUBS and Q-learning-eGUBS+Cmax. Theoretical and experimental results show that the proposed algorithms make this trade-off according to the configuration of the GUBS’ parameters (Code enabled on https://github.com/QlearningGubs/code).
SARA - A Generative AI for Legal Process Summarization Based on Chain of Density Prompt Engineering
Generative AI, particularly Large Language Models (LLMs), holds significant promise for enhancing judicial tasks, especially in automating the generation of legal cases during the sentencing phase. This paper introduces SARA (System for Analysis and summaRization of legal Actions), an innovative method for abstractive and multi-document summarization of legal proceedings. SARA employs GPT-4o, trained exclusively through in-context learning, utilizing Chain of Density (CoD) and CO-STAR prompt engineering techniques. These methods, adapted from judicial procedural knowledge, significantly improve the quality of generated summaries. Traditional evaluation metrics reveal effective training strategies but highlight their limitations in assessing summary quality. Therefore, we propose a qualitative evaluation methodology based on expert-generated questionnaires, focusing on essential content inclusion and proper report structuring. Inspired by this methodology, we trained another GPT model for large-scale summary evaluation. Evaluations of summaries of fifteen first-degree court cases from the Court of Justice of the State of Ceará show a significant advantage of in-context learning with CoD, emphasizing the role of domain knowledge and report style.
Semi-supervised Predictive Clustering Trees for Multi-label Protein Subcellular Localization
Protein subcellular localization is an important classification task because the location of proteins in a cell is directly linked to their functions. Since a protein can act at two or more locations simultaneously, multi-label classification algorithms are necessary. The currently used algorithms are usually based on supervised learning, which presents some disadvantages such as (i) a need for a large amount of labeled instances for training; (ii) a waste of valuable information that labeled instances can provide; and (iii) a high cost involved in obtaining labeled instances for training. To overcome these disadvantages, semi-supervised learning can be applied, where classifiers exploit both labeled and unlabeled data. Thus, in this paper, we propose a new semi-supervised algorithm for multi-label protein subcellular localization. Our proposal is based on decision tree classifiers induced using predictive clustering trees. We investigate many semi-supervised protein subcellular localization scenarios to test whether unlabeled instances can improve the multi-label classification process. Our results show that the proposal can achieve competitive or better results when compared to the pure supervised version of the predictive clustering trees.
Siamese Network-Based Prioritization for Enhanced Multi-document Summarization
Methods for document summarization hold significance in numerous applications, particularly in scenarios involving extensive content, such as news and social media monitoring. Nevertheless, enhancing the task of summarizing multiple documents remains an area for improvement. Systems that generate summaries from multiple documents must take into account additional challenges such as document redundancies and inconsistencies. On the other hand, as multi-document databases contain multiple descriptions of the same content, they offer benefits that go beyond the availability of more content and we believe that this benefit has not been explored in the literature. Different authors may write different descriptions of the same event, some more objective, others more detailed, or even with different terms or writing styles. As a way to take advantage of this availability of different descriptions, we present a new approach to evaluating documents in order to identify those that can contribute most effectively to the generation of automatic summaries. For this, we employed a Siamese network, which was trained using the ROUGE metric observed in individual documents. Additionally, we demonstrated how to apply the outcomes of document evaluation to different summarization techniques. Our experiments included a comparison with SOTA approaches (LeadSum, TextRank, PacSum, BertSum) in multiple datasets encompassing news, events, Wikipedia texts, and scientific publications (Multi-News, WCEP, WikiSum, arXiv, Multi-XScience) and the results indicated our approach had a relevant improvement in the production of summaries, with statistical significance in the evaluation with Wilcoxon rank sum for a confidence factor of 95%.
Although fundamental to the advancement of Machine Learning, the classic evaluation metrics extracted from the confusion matrix, such as precision and F1, are limited. Such metrics only offer a quantitative view of the models’ performance, without considering the complexity of the data or the quality of the hit. To overcome these limitations, recent research has introduced the use of psychometric metrics such as Item Response Theory (IRT), which allows an assessment at the level of latent characteristics of instances. This work investigates how IRT concepts can enrich a confusion matrix in order to identify which model is the most appropriate among options with similar performance. In the study carried out, IRT does not replace, but complements classical metrics by offering a new layer of evaluation and observation of the fine behavior of models in specific instances. It was also observed that there is 97% confidence that the score from the IRT has different contributions from 66% of the classical metrics analyzed.
SurveySum: A Dataset for Summarizing Multiple Scientific Articles into a Survey Section
Document summarization is a task to shorten texts into concise and informative summaries. This paper introduces a novel dataset designed for summarizing multiple scientific articles into a section of a survey. Our contributions are: (1) SurveySum, a new dataset addressing the gap in domain-specific summarization tools; (2) two specific pipelines to summarize scientific articles into a section of a survey; and (3) the evaluation of these pipelines using multiple metrics to compare their performance. Our results highlight the importance of high-quality retrieval stages and the impact of different configurations on the quality of generated summaries.
Traffic Forecasting Using Federated Randomized High-Order Fuzzy Cognitive Maps
Numerous machine learning (ML) and deep learning (DL) forecasting techniques have emerged for predicting traffic flow and traffic speed. Despite their notable achievements, they encounter challenges related to preserving in-vehicle user privacy. Alternatively, Federated Learning (FL) offers a decentralized ML strategy to strike a balance between prediction accuracy and privacy preservation. This paper proposes a novel fuzzy-based method called FL-RHFCM, which integrates the principles of Randomized High-order Fuzzy Cognitive Maps (R-HFCM) with FL. R-HFCM is akin to an echo state network (ESN), where the only trainable component is the output layer using least squares (LS) minimization. The reservoir layer’s weights are initialized randomly and remain unchanged during training. Thus, the LS coefficients represent the only parameters shared with the server for aggregation in our FL-RHFCM approach. FL-RHFCM introduces an efficient distributed forecasting method using FCMs, significantly reducing communication costs. The effectiveness of our proposed model is assessed using two datasets, demonstrating its promise compared to some existing baseline methods.
Unsupervised Statistical Keyword Extraction Pipeline: Is LLM All You Need?
Keyword extraction is an important step for text interpretation, serving to identify and highlight the most significant words or phrases within a text. This step is essential for various applications such as summarization, indexing, and information retrieval. This paper presents a custom-built keyword extraction pipeline named USKE (Unsupervised Statistical Keyword Extraction) and compares its performance to large language models (LLMs). USKE is able to deliver fast and simple results based in statistical methods even when dealing with large datasets. Our evaluation demonstrates that although LLMs can achieve good results in single sentences with minimal context, they require a lot of post-processing and may output inconsistent answers, while USKE excels in efficiency and scalability.
Using Complex Networks to Improve Legal Text Hierarchical Classification
Legal topics are typically organized into trees of labels, where each branch, from the root (generic topics) to the leaf (specific topics), categorizes the vocabulary that describe lawsuits. In this article, we describe an approach, innovative in the Brazilian Justice System, of automatic hierarchical classification of a petition topic. The approach integrates methods based on transformers and complex networks to capture, in addition to the characteristics of the text, the relationship between the legal citations present in the petitions and the topic to which the petition refers. The validation of this approach is done through a benchmark that shows accuracy gains, as well as, a practical scenario with the implementation of a microservice on a National Justice Platform whose front-end implementation is already being used by a State Court to automatically suggest the classification of the petitions topic in the National Procedural System.
A Case Study on Water Demand Forecasting in a Coastal Tourist City
Urban Water-Demand (UWD) forecasting is crucial for efficient water management, improving distribution, and supporting environmental sustainability. In tourist destinations with significant seasonal variations in number of inhabitants (water consumers), accurate water-demand forecasting becomes particularly important. This work evaluates two statistical models for short-term UWD forecasting, namely, Autoregressive Integrated Moving Average (ARIMA) and Seasonal Autoregressive Integrated Moving Average (SARIMA). Two different strategies for model deployment and comparison are considered: (i) a sliding window (SW) approach with one-year (1Y) and two-year (2Y) windows for training and; (ii) a expanding window (EW) approach. The ARIMA model ployed with a Sliding Window (SW) with a two-year (2Y) resolution achieved the best overall results, followed by SARIMA considering Expanding Window (EW) model. To place these outcomes in perspective, we performed a comparison with results from related work that took into account Machine Learning methods for regression for the same data. This comparison suggests that statistical methods provide results that are both competitive and robust in terms of quality for short-term forecasts.
A Knowledge Engineering-Based Approach to Detect Gaming the System in Novice Programmers
In recent years, the behavior called “gaming the system” has received increasing attention, especially in online learning environments, where students seek to advance in the system without actually engaging in genuine learning processes. Although previous studies have made progress in identifying and understanding this behavior, making use of machine learning models and knowledge engineering, previously validated detectors have not fully captured the gaming attitudes observed in our novice programming learning context. This challenge may be related to the transferability of motivations for gaming between different contexts, as well as to detection. Therefore, this article adopts an approach focused on the exploration and specific detection of “gaming the system” attitudes manifested by novice programmers in interaction with a learning environment. We propose an approach to detecting this behavior, combining knowledge engineering and machine learning algorithms. We describe the procedures for generating our database, followed by the elaboration of a model based on knowledge engineering and the development of a model based on machine learning. As a result, we identified two new game attitudes and incorporated them into the knowledge-based model, using cognitive task analysis to understand how experts encode these attitudes.
Patricia Cabral de Azevedo Restelli Tedesco
Acoustic Features and Autoencoders for Fault Detection in Rotating Machines: A Case Study
Traditional Machine Fault Detection (MFD) techniques usually rely on multiple sensor data sources, such as vibration, temperature, force, and audio/acoustic signals. Acoustic signals, in particular, are quite appealing in the context of MFD, as they are often among the first manifestations of machine failure. Furthermore, they are associated with high sensitivity, environmental resilience, and do not require machine interference. Given these compelling characteristics, MFD based exclusively on acoustic signals can be highly beneficial. In this work, we evaluate an unsupervised MFD approach based on Autoencoders (AE) trained exclusively on features extracted from acoustic signals of a rotating machine. The data employed in this work comes from the Machine Fault Database (MaFaulDa), which includes information from vibration and velocity sensors, besides the acoustic measurements. This allows us to compare the performance of the AE models to that of supervised models (such as MLPs) trained on the same acoustic-based feature set, as well as feature sets that incorporate all sensors from MaFaulDa. Our results support that unsupervised MFD based on Autoencoders and acoustic signals is particularly appealing, as it requires only normal machine operation for training. Indeed, we obtained AUC values of 0.86 for the task.
Affective States in Novice Programmers: Automatically Detecting and Analyzing the Impact on Learning
Programming is a complex cognitive activity that often poses a significant challenge for beginners. Beyond the cognitive dimension, understanding the role of affective states, such as emotions, attitudes, and motivation, in the learning process can provide valuable insights for educators to improve teaching methodologies and curriculum design. Despite this importance, few studies relate the cognitive aspects of beginning programmers to their affective states in the context of problem-solving. In this article, we explore the affective states reported by novice programmers in introductory programming courses, relating them to programming concepts and success rates during problem-solving activities. Furthermore, we apply machine learning algorithms to detect affective states in novice programmers, considering both student and task characteristics. Our initial results allow us to understand the affective states most prevalent in problems associated with each programming concept, as well as to reveal a significant association between affective states and the performance of novice programmers. Positive affective states, including pleasure, motivation, and challenge, were positively correlated with better performance outcomes. On the other hand, negative affective states such as frustration, anxiety, and boredom impaired performance. Additionally, our algorithms demonstrated good performance in detecting affective states.
An Analysis of Time-Frequency Consistency in Human Activity Recognition
This work relies upon raw data to present the time-frequency consistency (TF-C) evaluation for human activity recognition (HAR). The original paper utilized data for this task in the pretext stage but did not explore its application in the downstream task. An application with a modified TF-C architecture uses HAR data on the downstream task, reporting an accuracy of 64.08%. We propose three experiments. First, we reproduce the original experiment with the epilepsy dataset, comparing the results with the reported ones. Second, we make a performance comparison test using different percentages of data from 0.1% to 100% and report the corresponding accuracy. Finally, we compare the results with supervised Convolutional Neural Networks and the supervised TF-C. This work demonstrates the feasibility of utilizing TF-C to perform HAR as downstream task, achieving an accuracy of 96% utilizing all data of the training dataset in fine-tuning. Even with just 42 samples of the training dataset, the model achieved an accuracy of 85% and to obtain an accuracy greater than 90% it is only necessary 126 train samples.
An Evaluation of Temporal Neighborhood Coding Variants in Smartphone-Based Human Activity Recognition
Self-Supervised Learning (SSL) has emerged as a powerful tool for learning valuable representations from large amounts of unlabeled data. Human Activity Recognition (HAR) is a field that may benefit from SSL techniques, as there is a large amount of unlabeled data, and labeling is time-consuming and costly. Temporal Neighborhood Coding (TNC) is a SSL technique that shows promise in extracting meaningful features automatically. However, the effectiveness of different TNC variations for HAR data has yet to be comprehensively evaluated. Current research focuses on specific contexts, leaving a gap in understanding how these variations perform across the same dataset. Additionally, it is necessary to assess the impact of applying TNC on raw data instead of on handcrafted features. This paper systematically evaluates different variations of TNC for HAR, investigating their performances in a standardized quantitative and qualitative approach, focusing on raw data but comparing with featured engineered data of the same dataset. Our findings show that using the dilated convolution encoder proposed by TS2Vec is currently the best alternative in terms of performance, achieving 95% accuracy in the UCI dataset with raw data. Additionally, we verified that by replacing the Augmented Dickey-Fuller (ADF) statistical test with the cosine similarity to select neighboring windows makes training time seven to nine times faster with little performance impact. Finally, we found that learning from handcrafted features from UCI dataset is easier, but advanced versions of TNC can effectively learn robust features from raw data, achieving performance comparable to models trained on the handcrafted features.
AnisotropicBreast-ViT: Breast Cancer Classification in Ultrasound Images Using Anisotropic Filtering and Vision Transformer
Breast cancer classification in ultrasound images is a challenging and arduous task, primarily due to the quality of the images and the complexity of the lesions encountered. Early diagnosis of this pathology is known to increase patient survival chances. Therefore, computational methods have been developed to assist specialist physicians in this critical task. This study introduces AnisotropicBreast-ViT, a method that integrates anisotropic filtering, balanced data augmentation, and Vision Transformer to aid in the classification of breast ultrasound images. The proposed approach achieves promising results with an accuracy of 98.82%, specificity of 98.62%, sensitivity of 99.25%, precision of 97.08%, F1-score of 98.16%, and AUC-ROC of 0.989, surpassing current benchmarks in the field. These findings suggest that AnisotropicBreast-ViT has the potential to significantly improve breast cancer diagnosis, demonstrating its effectiveness and robustness in clinical applications.
Anomalies Diagnostic in Endoscopic Images Using Deep Learning Ensemble Models
Automated anomaly detection in medical images greatly benefits the medical community by assisting in the diagnosis and identification of clinical cases. This work focuses specifically on medical images obtained from endoscopies. We categorized the images into two classes: normal (including normal-cecum, normal-pylorus, normal-z-line) and abnormal (including dyed-lifted-polyps, dyed-resection-margins, esophagitis, polyps, ulcerative colitis). We propose data augmentation to balance the classes and deep learning model techniques for classification. We employed an Ensemble Voting method with deep learning models to enhance the accuracy of the process. Our results are robust, achieving an accuracy of 98.62%, a precision of 96.47%, a sensitivity of 99.64%, and an F1-score of 98.03%. This approach proves to be an effective tool for identifying anomalies in endoscopies. We believe our method can contribute to significant advancements in AI-assisted medical diagnostics.
Automated Segmentation of Computed Tomography Images for COVID-19 Patient Evaluation
COVID-19, caused by SARS-COV-2, resulted in 774 million cases and 7 million deaths by March 2024. This study proposes an approach to detect pulmonary lesions in computed tomography scans, integrating classification, preprocessing, and segmentation. Initially, a model based on LeNet-5 classifies the relevant slices of the scans, eliminating the irrelevant ones. Subsequently, the selected images undergo contrast adjustments, binarization, and normalization. Afterwards, segmentation is performed using a U-Net-based architecture, allowing for detailed segmentation. The methodology achieved 78.40% Dice, 64.80% IoU, 78% Sensitivity, 100% Specificity, 89.60% AUC, and 81% Precision, using only 9 million parameters. These results offer a practical and efficient solution, supporting specialists in patient treatment.
Combining Clustering and Genetic Algorithms for Portfolio Optimization: A Case Study with B3 Companies
Portfolio optimization has been addressed by optimization algorithms and due to the numerous local maxima, genetic algorithms have shown promise. Recent studies use these algorithms to optimize stock portfolios, whether as a multi-objective problem that maximize return and minimize risk or just as one objective problem, such as maximizing the Sharpe Ratio. In order to adapt these algorithms to the financial market, a new approach was developed that includes penalty and cardinality constraints. This study proposes an algorithm with two models of clustering of B3 companies with penalties and cardinality constraints. One model uses the clustering of companies by economic sector and the other model uses the clustering generated by the Kmeans-DTW time series clustering technique. Then the algorithm selects one asset per cluster of companies with the aim of diversifying the portfolio and reducing the correlation between assets, thus reducing the risk. Finally, the generated portfolios went through an optimization stage using a memetic genetic algorithm, which finds the best possible distribution that maximizes return and reduces risk.
Comparative Analysis of Machine Learning Algorithms for Identifying Genetic Markers Linked to Alzheimer’s Disease
The identification of genetic markers for complex diseases like Alzheimer’s Disease (AD) is pivotal in medical genomics. This study aims to identify genetic markers associated with AD by introducing a novel approach that exclusively utilizes genetic data. Our primary goals are to benchmark explainable machine learning models against BLUPF90, an advanced mixed linear model approach, and to uncover single nucleotide polymorphisms (SNPs) crucial for AD. We analyze SNPs to achieve these goals, focusing on the genetic heritability rate of 58–79% for AD [12]. Our methodology focuses solely on genetic data to uncover SNPs crucial for AD, employing transparent computational models to ensure interpretability alongside predictive power. The findings demonstrate the efficacy of a purely genomic approach combined with Machine Learning to advance our understanding of AD. Our methodology successfully identified a robust set of SNPs associated with AD, encompassing both previously recognized and novel SNPs. The Machine Learning models employed delineated distinct SNP profiles, highlighting the complexity and heterogeneity of AD. These results not only deepen our understanding of AD’s genetic underpinnings but also facilitate the development of targeted therapeutic and diagnostic strategies, showcasing the potential of computational techniques in medical genomics.
Alzheimer’s Disease Neuroimaging Initiative
Alzheimer’s Disease Neuroimaging Initiative Comparing LIME and SHAP Global Explanations for Human Activity Recognition
The development of complex machine learning models has been increasing in recent years, and the need to understand the decisions made by these models has become essential. In this context, eXplainable Artificial Intelligence (XAI) has emerged as a field of study that aims to provide explanations for the decisions made by ML models. This work presents a comparison between two state-of-the-art XAI techniques, LIME and SHAP, in the context of Human Activity Recognition (HAR). As LIME provides only local explanations, we present a way to compute global feature importance from LIME explanations based on a global aggregation approach and use correlation metrics to compare the feature importance provided by LIME and SHAP across different HAR datasets and models. The results show that using correlation metrics to compare XAI techniques is not enough to conclude if the techniques are similar or are not, so we employ a feature removal and retrain approach and show that, besides some divergences in the correlation metrics, both XAI techniques successfully identify the most and least important features used by the model for the task.
Damage Identification of Wind Turbine Blades
Wind turbines capture the kinetic energy produced by the wind, with the blades being the component most susceptible to damage. Unplanned stops result in significant losses, which highlights the need to detect these failures early. As a step in the preventative maintenance procedure, hundreds of color photographs of the blades are taken for subsequent analysis by an expert. In this work, we present a method to highlight superficial damages in wind turbine blades using a classification and localization process. A new dataset was created using images of wind blades, each divided into uniform slices and labeled by an expert according to the type of fault identified. Then, we apply class balancing and data augmentation methods prior to fine-tuning a general-propose pre-trained deep convolutional neural network model. The best model was used to classify and locate damages in the wind blades images. As a result, we obtained an overall precision of 96.1%, accuracy of 97% and recall of 94.5% when classifying the presence of damages. We show that our method can be integrated into the monitoring of wind blade tasks, helping the specialist to highlight and identify images containing damage.
Emotion Recognition in Instrumental Music Using AI
This study aims to develop a predictive model for classifying emotions in instrumental music excerpts using machine learning techniques. The methodology involves experiments with three distinct datasets, including a newly created dataset for this research. The models employed include Random Forest, Multi-Layer Perceptron, and Convolutional Neural Network architectures. The experimental results, conducted with a single dataset for testing and validation, were generally positive. However, when attempting to generalize the models using different datasets, a considerable reduction in generalization capability was observed. Despite this, the study presents promising results, indicating that increasing the number of training data can significantly improve the models’ generalization capability.
Enhancing Multiobjective Genetic Algorithms for Pharmaceutical Batch Scheduling: A Study on Partitioned Selection with Constraints and Mutation with Greedy Local Search Strategy
Product Manufacturing scheduling or Batch Sequencing (BS) in real-world pharmaceutical manufacturing poses a complex multiobjective optimization challenge, intensified by constraints and uncertain demands. These problems are typically non-linear, may leading to low convergence of feasible solutions, and the computationally intensive evaluation functions may hinder the adoption of certain techniques. Multiobjective Evolutionary Algorithms (MOEAs) have been explored due to their flexible and efficient operators that can be applied to various non-linear multiobjective problems. This study proposes enhancements to MOEA operators, specifically a selection strategy with constraint-based partition for reinsertion to improve diversity and exploration, and a greedy local search strategy incorporated into the mutation operator to enhance the exploitation of the most promising regions. These enhancements were integrated into two models, ps-re and bl-bat, both of which demonstrated significant improvements in the performance metrics compared to a reference model.
Evaluating Sentiment Quantification Methods in Brazilian Portuguese Corpora
This paper evaluates sentiment quantification methods applied to Brazilian Portuguese corpora. Sentiment quantification, distinct from sentiment classification, estimates the distribution of sentiment classes (positive and negative) within a dataset. We investigate several quantification techniques, including the family Classify and Count (CC) and more sophisticated methods, such as Kernel Density Estimation (KDE) and Distribution y-Similarity (DyS). Our analysis uses five datasets, each containing different distributions of sentiment classes. Our experimental results indicate that KDE and DyS methods consistently outperform others, achieving the best average ranks in terms of quantification accuracy. Statistical tests, including the Friedman and Nemenyi tests, confirm significant performance differences among the methods, with KDE and DyS showing statistically significant improvements over the baseline CC method. These findings highlight the importance of choosing robust quantification techniques for accurate sentiment quantification in corpora across different domains.
Evaluating Short Text Stream Clustering on Large E-commerce Datasets
Latent Dirichlet Allocation (LDA) is a fundamental method for clustering short text streams. However, when applied to large datasets, it often faces significant challenges, and its performance is typically evaluated in domain-specific datasets such as news and tweets. This study aims to fill this gap by evaluating the effectiveness of short text clustering methods in a large and diverse e-commerce dataset. We specifically investigate how well these clustering algorithms adapt to the complex dynamics and larger scale of e-commerce text streams, which differ from their usual application domains. Our analysis focuses on the impact of high homogeneity scores on the reported Normalized Mutual Information (NMI) values. We particularly examine whether these scores are inflated due to the prevalence of single-element clusters. To address potential biases in clustering evaluation, we propose using the Akaike Information Criterion (AIC) as an alternative metric to reduce the formation of single-element clusters and provide a more balanced measure of clustering performance. We present new insights for applying short text clustering methodologies in real-world situations, especially in sectors like e-commerce, where text data volumes and dynamics present unique challenges.
Evolutionary Adjustment of a Cellular Automata-BasedModel for Wildfire Spreading
Due to climate change, the incidence of fires has increased considerably in recent decades, causing various damages to the environment. In this scenario, computational models based on Cellular Automata (CA) are capable of reproducing and predicting the behavior of fire propagation, aiding in control and prevention measures. However, constructing and adjusting the parameters of these models is not a trivial task. In this context, a method capable of automatically adjusting these parameters can assist in this stage, which demands deep knowledge on the subject. In this work a Genetic Algorithm (GA) is presented aiming the adequate adjusting of a CA-based model parameter set. Additionally, a new parameter is proposed allowing the adaptability of the CA model to the fire spreading dynamics, regardless of the sampling rate used to generate the reference data. The experiments shown that our approach was able to capture the dynamics present in the data.
Exploring Score-Based Ranking Fairness in Marketplace Environments Through Simulation
We are currently witnessing growing concerns regarding fairness in online marketplace environments, particularly in situations involving platform-owned first-party selling, biased recommendation algorithms and trust formation. Regulatory measures, such as those imposed by the European Commission and the US Federal Trade Commission, underscore the importance of fair practices on these platforms. While machine learning ranking models are widely used in ranking multiple offers from various sellers, they can inadvertently introduce biases, raising regulators’ concerns. Despite recent advances in fair ranking algorithms, the long-term impact of fairness in ranking remains understudied. The contributions of this research are threefold: 1) it establishes a simulated online marketplace environment to complement traditional static experiments; 2) it assesses the long-term impact of fairness in utility of ranking algorithms; and 3) it evaluates state-of-the-art fairness techniques in dynamic environments. Findings reveal how utility in ranking fairness algorithms can be affected by the application of fairness techniques and how data drift impacts regular and fair ranking algorithms in a long-term scenario.
HAVANA: Hybrid Attentional Graph Convolutional Network Semantic Venue Annotation Model
The increasing geospatial data availability has enabled extensive urban mobility studies. However, some tasks require point-of-interest labels, which are missing or inaccurate on public datasets. In this context, the existing solutions fail to utilize different types of convolutional filters regarding graph neural networks, thereby hindering their performance in labeling place categories. To address these shortcomings, this work proposes a new model to annotate points of interest semantically characterized by a hybrid architecture that uses a spatial and spectral filter integrated with a self-attention mechanism. Our results demonstrate an improvement of up to 25.05% in F1-Score compared to three state-of-the-art models across three different datasets.
Impact of Pre-training Datasets on Human Activity Recognition with Contrastive Predictive Coding
Self-Supervised Learning (SSL) techniques have been successfully employed to learn useful representations for various data modalities without labels. These techniques use a pretext task to train the backbone of a deep-learning model without labels and then leverage the pre-trained backbone to train a downstream model with a few labeled samples. In this context, Contrastive Predictive Coding (CPC) is an SSL technique that has demonstrated promising results in several tasks, including human activity recognition (HAR). In this work, we explore the impact of data variety on backbone pre-training when designing CPC models for HAR and the benefits of pre-training on the final model. We evaluated the impact of data variety on model pre-training using fifteen combinations of four distinct HAR datasets, finding significant performance variability based on the pre-training datasets, with F1-score varying from 9.6 to 13% points across different target datasets. We also found that including the target dataset in the pre-training process generally improved performance and that pre-training with all four datasets produced a high-quality backbone, yielding downstream models performing near the best on all target datasets. These findings emphasize the importance of selecting pre-training datasets aligned with the downstream task domain. Additionally, we demonstrated that CPC pre-training significantly benefits downstream model performance with limited data, achieving comparable F1-scores with just 5% of the data as with 100%, indicating that CPC effectively captures essential features of the problem domain.
Improving Colorectal Cancer Diagnosis Using MIRNet and InceptionV3 on Histopathological Images
Colorectal cancer (CRC) is the second most prevalent type of cancer among men and women in Brazil, accounting for 9.1% of cancers in men and 9.2% in women between 2020 and 2022. CRC diagnosis typically relies on histopathological images, which can be challenging to interpret and prone to errors. This paper proposes a method for diagnosing CRC using histopathological images. This method involves enhancing images with MIRNet and classifying them as benign or malignant using InceptionV3. The results surpass the state-of-the-art by introducing, for the first time, enhancement through MIRNet, achieving an accuracy of 99.67% and an AUC of 0.9999. We believe that this method, combined with medical expertise, holds great promise for early CRC diagnosis.
Integrating Tensor-Based Data Analytics and Adaptive Prediction for Informed Decision-Making Support
This work proposes a novel approach to support multi-criteria decision analysis (MCDA) using tensor-based data structures and an adaptive prediction method. MCDA allows for informed decision-making involving the evaluation of different alternatives based on a set of predefined criteria. Unlike previous approaches, this methodology considers the prediction of future criteria signals rather than just consider a single-period value for the criteria. The proposed method generates a tensorial representation of the data and ranks alternatives using a MCDA method. Experimental results demonstrate that this approach outperforms existing methods, particularly in decision-making situations where future long-term consequences need to be considered. This study contributes to the development of decision support systems by providing a methodological framework that leverages the potential of signal processing and tensor-based data analysis.
Investigating Methods to Detect Off-Topic Essays
Automated Essay Scoring is one of the most important educational applications of natural language processing. It helps teachers with automatic assessments, providing a cheaper, faster, and more deterministic approach than humans when scoring essays. Nevertheless, off-topic essays pose challenges in this area, causing an automated grader to overestimate the score of an essay that does not adhere to a proposed topic. Thus, detecting off-topic essays is important for dealing with unrelated text responses to a given topic. This paper explored approaches based on handcrafted features to feed supervised machine-learning algorithms, tuning a BERT model, and prompt engineering with a large language model. We assessed these strategies in a public corpus of Portuguese essays, achieving the best result using a fine-tuned BERT model with a 75% balanced accuracy. Furthermore, this strategy was able to identify low-quality essays.
Machine Learning and Time Series Analysis to Forecast Hotel Room Prices
The hospitality industry’s dynamic nature demands accurate forecasting of hotel room prices to optimize revenue management strategies. This paper presents an experimental study assessing machine learning techniques and time series analysis for forecasting hotel room prices. We enhance prediction accuracy by leveraging historical booking data, seasonal patterns, and hotel characteristics. We employ time series models, including AutoRegressors and Prophet, to capture underlying trends and seasonal variations. We also evaluate machine learning models such as random forest, gradient boosting machine, extra trees regressor, and neural networks. These models are trained on features like booking lead time, historical hotel occupancy, room nights, and number of adults to capture complex relationships influencing prices. Our methodology is demonstrated through a case study using over 40,000 reservations from a Brazilian hotel over a decade. Experimental results show that tree-based models performed best, with the Gradient Boosting achieving 6.94% of NRMSE and 31.26 of MAE. Our findings contribute valuable insights for price estimation in the hospitality industry, offering robust methods to enhance revenue management strategies.
Modeling and Predicting Crimes in the City of São Paulo Using Graph Neural Networks
Crime prediction is a critical research area for enhancing public safety and optimizing law enforcement resource allocation, and machine learning techniques have had a significant impact in this field. Traditional machine learning models have long struggled to capture complex crime patterns, primarily due to the intricate interdependence of spatial and temporal data. However, recent advancements in machine learning, particularly with Graph Neural Networks (GNNs), offer a new perspective. GNNs have demonstrated remarkable success in various applications and they can also play a significant role in crime analysis and prediction. Therefore, in this work, we explore such a potential by examining two distinct spatiotemporal GNN architectures, namely Dynamic Self-Attention Network (DySAT) and Evolving Graph Convolutional Network (EvolveGCN), assessing and comparing their effectiveness for crime prediction. Moreover, we propose a data modeling framework that integrates crime, street map graphs, and urban data, which is fundamental to properly train the GNN models. As far as we know, there is no consolidated methodology to integrate those three modalities of data, being a relevant contribution of this work. Our findings underscore the effectiveness of GNNs in crime prediction tasks, offering valuable insights for researchers and practitioners in the field of crime prevention and public safety enhancement.
Predicting Engagement of Brazilian Politicians on TikTok: A Machine Learning Approach
While established social media platforms like Facebook, Twitter, and Instagram have become staples in political campaigns, the 2022 Brazilian elections witnessed the rise of a new contender: TikTok. Despite its recent emergence in 2016, TikTok has already become the fourth most used social network in Brazil. This study investigates the potential of machine learning to predict engagement on the TikTok profiles of the two leading presidential candidates: Lula and Bolsonaro. Utilizing a dataset from previous studies, we implemented various machine learning models and found that the Support Vector Machine achieved the highest performance based on the F1-score metric for both candidates. Despite the results being better with Bolsonaro than with Lula, further analysis of metrics like recall and precision suggests valuable insights for social and political domains. These findings can aid both candidates and society in understanding what factors are most related to engagement on this emerging social media platform. Additionally, marketing and advertising teams can use this information to create content tailored to reach and engage with a politician’s target electorate.
Preserving Privacy, Enhancing Robustness: Federated Learning for Lung Disease Identification in Chest X-Ray Images
While hospitals routinely gather patient data, such as X-ray images, the challenge of sharing this data across multiple institutions to create a comprehensive and large dataset is hampered by privacy concerns. Consequently, this limitation affects the effectiveness of state-of-the-art deep neural networks for tasks like identifying lung diseases in medical images, as they require substantial annotated data. Federated Learning offers a solution by enabling collaborative training across multiple edge devices or sites, where updates (e.g., neural network weights) are aggregated without sharing patient data, thus maintaining privacy. This work introduces a federated-learning-based approach for automatically detecting lung diseases in chest X-ray images, focusing on preserving data privacy and enhancing robustness. Our approach follows the federated learning protocol: decentralized training of neural networks on data from multiple sites (hospitals) and centralized aggregation of knowledge in the server. The solution presents promising results in identifying fourteen lung diseases compared to three baselines within a simulated environment comprising chest X-ray images from five distinct sites.
RLPortfolio: Reinforcement Learning for Financial Portfolio Optimization
Portfolio optimization is a task in which an agent constantly rebalances a predefined portfolio of assets in order to mitigate losses and maximize profits. A very effective way to solve this task is using a reinforcement learning agent that learns an optimal investment strategy by interacting with the environment, but due to the current lack of open source tools to accelerate the development of such an agent, its implementation is quite difficult. Therefore, to reduce this obstacle, this paper introduces RLPortfolio, a Python library which provides the necessary tools to develop, train and evaluate reinforcement learning agents whose function is to optimize financial portfolios over time. The library contains a simulation environment that implements the state-of-the-art mathematical formulation of the portfolio optimization task, a policy gradient algorithm developed specifically to train agents to solve this task and four state-of-the-art deep neural network architectures that can be used as the agent’s action policy. This paper also demonstrates the results of an agent trained with RLPortfolio to optimize a financial portfolio composed of ten high-volume stocks from the Brazilian market, proving the reliability of the solutions contained in the library while also showing that it produces agents that perform better than classical approaches.
Scaling and Adapting Large Language Models for Portuguese Open Information Extraction: A Comparative Study of Fine-Tuning and LoRA
This paper comprehensively investigates the efficacy of different adaptation techniques for Large Language Models (LLMs) in the context of Open Information Extraction (OpenIE) for Portuguese. We compare Full Fine-Tuning (FFT) and Low-Rank Adaptation (LoRA) across a model with 0.5B parameters. Our study evaluates the impact of model size and adaptation method on OpenIE performance, considering precision, recall, and F1 scores, as well as computational efficiency during training and inference phases. We contribute to a high-performing LLM and novel insights into the trade-offs between model scale, adaptation technique, and cross-lingual transferability in the OpenIE task. Our findings reveal significant performance variations across different configurations, with LoRA demonstrating competitive results. We also analyze the linguistic nuances in the Portuguese OpenIE that pose challenges for models primarily trained on English data. This research advances our understanding of LLM adaptation for specialized NLP tasks and provides practical guidelines for deploying these models in resource-constrained and multilingual scenarios. Our work has implications for the broader cross-lingual open information extraction field and contributes to the ongoing discourse on efficient fine-tuning strategies for large pre-trained models.
Special-Crowd-Distance Boosted MESH Applied to the Operation of Cascade Hydro-Power Plants
Most of Brazil’s and China’s electricity comes from renewable sources through hydroelectric power plants. The operation of these plants can be modeled as a multi-objective optimization problem, in which the generation of energy, the water level in the reservoirs, and the minimal alteration of water flow after the optimization process are maximized. In this context, the work proposes to study the performance of different evolutionary meta-heuristics applied to the problem of electric dispatch in cascade mode power plants (at least two plants on a riverbed). The work has a special focus on the Multi-Objective Evolutionary Swarm Hybridization (MESH) algorithm boosted by the Special-Crowd-Distance operator, to solve three conflicting objectives in an Operation Multi-Reservoir System (OMRS). Experiments in this work indicate that MESH version proposed here is competitive when compared to standard evolutionary meta-heuristics: the Multi-Objective Evolutionary Algorithm Based on Decomposition (MOEA/D), versions of Non-dominated Sorting Genetic Algorithm (NSGA II and III), and the standard MESH solving the real-world problem here addressed.
The Impact of Double Transfer Learning in VGG Architectures for Metastasis Breast Cancer Detection
According to the World Health Organization (WHO), 2.3 million women were diagnosed with breast cancer in 2020, causing almost 700.000 deaths worldwide. The first occurrences (in situ stage) usually have a good response if there is an early detection because the earliest form of tumor does not have sufficient potential to kill. However, the next stage has a low survival rate because the most threatening tumor characteristic is cell division spreading throughout the body, damaging lungs, livers, bones, and the brain. An alternative to deal with this problem is to enhance the velocity of the diagnosis and detection of breast cancer. Thus, machine learning algorithms have proven to be effective in this task. In this context, this work investigates how double transfer learning improves two Deep Learning architectures, VGG-16 and VGG-19, using histopathological images, i.e., employing BreakHIS as the first transfer learning and PatchCamelyon as the second. Results indicate that using double transfer learning, results for precision, specificity, recall, and F-Score improve to 99%, 83%, 89%, and 90%, respectively.
A Topology-Inspired Approach to AGM Belief Change
AGM’s belief revision is one of the main paradigms in the study of belief change operations. Despite its popularity, it is well recognised that AGM’s work relies on a strong idealisation of the agent’s capabilities and on the supporting logic. Recent results in the literature have shed light on the definability of AGM-like belief change operations in general logics, showing how the properties of a logic can be leveraged to construct these operations. Exploring well-known connections between Logic, Algebra and Topology, our work explores a general notion of belief change contraction, which can be connected to well-studied operations in the literature, such as AGM rational contractions, partial meet contractions, and multiple contractions, employing the framework of Abstract Model Theory and Topological Semantics. We show that AGM rational contractions can be characterised in our framework, and we can establish connections to different results in the literature.
A Transformer-Based Tabular Approach to Detect Toxic Comments
In recent years, there has been a significant increase in toxic and hateful speech on social media platforms, becoming deeply entrenched in online interactions. This issue has drawn the attention of researchers from various academic fields, leading them to extend their focus to include disciplines such as Natural Language Processing, Machine Learning, and Linguistics, in addition to traditional areas like Law, Sociology, Psychology, and Politics. This paper introduces an approach for detecting toxic and hateful speech on social media using Tabular Deep Learning. The goal is to apply and evaluate the performance of the FT-Transformer model in detecting hateful and toxic content in textual comments on social media in Brazilian Portuguese. An important aspect of this research involves using modern embedding models as language embedders and language models and evaluating their performance with the FT-Transformer, a transformer-based tabular model. The experimental scenario uses a binary version of the ToLD-Br dataset. Our approach achieved a 76% accuracy rate and a 75% macro F1-score using the OpenAI text-embedding-3-large model.
Automatic Text Simplification for the Legal Domain in Brazilian Portuguese
Legal and juridical documents such as rulings, laws, agreements, and contracts contain domain-specific terms and jargon, long and complex sentences that may be difficult to understand for laypeople without domain expertise, reading issues, or with a low education level. The simplification of these documents has been a concern for several years, aiming to democratize access to justice. Courts are already adopting simpler language, especially in documents aimed at laypeople, such as warrants and notifications, to enhance inclusion and clarity. Automatic textual simplification, a subfield of Natural Language Processing, seeks to make complex texts more accessible. This paper explores the task of automatic text simplification in Portuguese for the legal domain. The main challenge here is the lack of datasets containing complex sentences and their simplified versions. This work investigates how existing datasets, methods, and metrics used for text simplification perform applied to legal texts in Portuguese. We present qualitative and quantitative analyses using five models. The results show that GPT-based models have the best results, but fine-tuning with domain data is a viable open-source alternative.
Developing Resource-Efficient Clinical LLMs for Brazilian Portuguese
In this study, we developed and evaluated two medical large language models, Clinical-BR-LlaMA-2-7B and Clinical-BR-Mistral-7B-v0.2, specifically designed for Brazilian Portuguese. Utilizing the Low-Rank Adaptation (LoRA) technique, our models achieved significant improvements in generating synthetic clinical text, particularly in terms of Authenticity of Format and Structure, Spelling Accuracy, and Clinical Coherence. The evaluation, conducted by medical students using a 5-point Likert scale, demonstrated the effectiveness of our approach compared to baseline models. The scores indicate superior performance compared to baseline models such as LlaMA-2-7B and Mistral-7B-v0.2. Our results suggest that these resource-efficient models can effectively generate clinically relevant text, maintaining high standards of structure, accuracy, and coherence. Future work will focus on expanding datasets, refining evaluation protocols, and enhancing model robustness to further improve performance across various medical tasks.
One-Class Learning for Data Stream Through Graph Neural Networks
In many data stream applications, there is a normal concept, and the objective is to identify normal and abnormal concepts by training only with normal concept instances. This scenario is known in the literature as one-class learning (OCL) for data streams. In this OCL scenario for data streams, we highlight two main gaps: (i) lack of methods based on graph neural networks (GNNs) and (ii) lack of interpretable methods. We introduce OPENCAST (One-class graPh autoENCoder for dAta STream), a new method for data streams based on OCL and GNNs. Our method learns representations while encapsulating the instances of interest through a hypersphere. OPENCAST learns low-dimensional representations to generate interpretability in the representation learning process. OPENCAST achieved state-of-the-art results for data streams in the OCL scenario, outperforming seven other methods. Furthermore, OPENCAST learns low-dimensional representations, generating interpretability in the representation learning process and results.
Semi-periodic Activation for Time Series Classification
This paper investigates the lack of research on activation functions for neural network models in time series tasks. It highlights the need to identify essential properties of these activations to improve their effectiveness in specific domains. To this end, the study comprehensively analyzes properties, such as bounded, monotonic, nonlinearity, and periodicity, for activation in time series neural networks. We propose a new activation that maximizes the coverage of these properties, called LeakySineLU. We empirically evaluate the LeakySineLU against commonly used activations in the literature using 112 benchmark datasets for time series classification, obtaining the best average ranking in all comparative scenarios.
José Gilberto Barbosa de Medeiros Júnior
Automated and Intelligent Vocational Guidance System for Classifying Specialties Based on POSCOMP Microdata
The National Examination for Admission to Postgraduate Studies in Computing (POSCOMP) is administered by the Brazilian Computer Society (SBC) to assess candidates’ knowledge for postgraduate programs in Computer Science in Brazil. Being one of the main evaluative instruments, the results of the POSCOMP exam, i.e., the database, can potentially obtain relevant patterns about the participants. In this sense, this article aims to propose an automated and intelligent vocational guidance system. This vocational system guides the participants to follow a research specialty based on the performance obtained in the exam. The proposed approach aims to support students in their academic decisions.
Classifying Potentially Non-compliant Portuguese Language Sentences Concerning Privacy Policies
Privacy policies (PPol) are extensive and contain complex sentences that are difficult to understand. They detail what happens with a person’s personal data and must comply with specific legislation. One way to assess the compliance of PPol with legislation is through machine learning models. There are some studies already carried out, aiming to detect compliance with the GDPR, basically analyzing PPol in English. In this work, we present a mapping of Brazilian data protection legislation into 27 categories, divided into 3 blocks, and 3 levels of potential compliance. We also introduced a corpus in Portuguese, with PPol sentences annotated through mapping of Brazilian legislation. We evaluated some classifier models in a task of detecting potentially non-compliant sentences and another task of categorizing potentially non-compliant sentences. We achieved performance close to the literature for studies in English and the GDPR. Our study points out ways to improve the automated assessment of PPol and highlights the complexity of the tasks that seek to ensure compliance with data protection legislation.
Diversity in Data for Speech Processing in Brazilian Portuguese
Striving to attend AI ethical guidelines is essential when developing and testing AI systems in order to ensure safe and trustworthy applications. However, these guidelines can be too general. The analysis presented here concerns the ethical principle of diversity, by discussing its application to the field of speech processing, using the task of prosodic segmentation of spontaneous speech as a case study. Particularly, it covers the relevance of including diversity of speaker’s profiles and regional variants in data used for training and developing AI applications, in the context of Brazilian Portuguese (BP). The contributions brought by this study are: (i) a discussion of the application of the diversity principle in the context of corpora for speech applications, considering some relevant aspects and the process we formulated to select a diverse sample of speakers to compose our corpus; (ii) a literature review of the current scenario of available corpora for the task of prosodic segmentation of spontaneous speech in BP, focused on the diversity of the data; (iii) a publicly available speech corpus (The corpus is publicly available in our Github repository https://github.com/nilc-nlp/MuPe-Diversidades/ under the CC BY-NC-ND 4.0 license) containing 2 h 32 min 15 s of spontaneous speech audios in BP, their revised transcriptions with automatic prosodic segmentation annotation, elaborated to comprise diversity of age, gender, and accents (17 Brazilian states).
Elevating Healthcare AI: Achieving Efficiency and Accuracy in Medical Applications with Surrogate-Based Multiobjective Compression of ResNet50 CNNs
Machine learning, particularly convolutional neural networks (CNNs), has gained prominence in healthcare applications, including medical diagnosis and clinical support. However, the increasing size of CNNs poses challenges in resource-constrained medical devices and real-time applications. This paper explores the effectiveness of pruning and quantization on the ResNet50 model within the MedMNIST dataset, a valuable resource for medical image classification. The study evaluates a surrogate-based multiobjective compression method on three MedMNIST datasets: RetinaMNIST for diabetic retinopathy grading, DermaMNIST for disease categorization, and BloodMNIST for blood cell classification. Results demonstrate that the proposed compression method successfully identifies less computationally intensive models while maintaining or improving accuracy across all three healthcare-related datasets. A reduction of about 50% in inference time and an increase of more than 1% in accuracy were observed. These findings emphasize the practicality of compression techniques in healthcare applications, particularly for resource-constrained environments and real-time decision-making scenarios. This research opens avenues for further validation and exploration in other healthcare-related applications with higher-quality neural network models, ultimately enhancing the deployment of machine learning in the healthcare domain.
Explainability of Machine Learning Models with XGBoost and SHAP Values in the Context of Coping with Disasters
A considerable part of Brazilian municipalities experiences the recurrence of emergencies, related to different types of disasters, which weakens them – both the public administration and the citizens – while they are equally oriented to operate transitions to achieve the 2030 Sustainable Development Goals (SDG) Agenda. To mitigate the crises addressed by the 2030 Agenda, intersectoral consultation guided by an integrated approach to social justice, accountability and sustainability is required to renew social dynamics and corresponding local public policies. The objective of this paper is to carry out an integrated analysis of such dynamics in their main social and economic components, modeling the indicators of some specific SGD and emergence decrees data as input variables to a classification problem and using explainable machine learning, namely SHAP values technique, to assess their intertwining and/or synergies. Three classification models were tested on data from 2003 to 2020 and the model which presented the best accuracy was analyzed with SHAP values, revealing that different variables are decisive for each phenomenon associated with the disaster and their understanding allows elucidating critical points to be addressed by managers.
Exploring Climatic Shifts in Brazilian Climates: Insights from ARMAX, Decision Trees, and Artificial Neural Networks
Over the years, humanity has deepened its understanding of its actions’ profound impacts on global temperatures. Beyond the rise in global temperatures driven by global warming, local biomes also significantly influence regional temperature variations, often due to changes in land use. Historically, climatologists have attributed local temperature increases to deforestation, urban development, and agricultural activities. Although these factors are well-known, there remains a need for methodologies to objectively quantify the impact of land use changes on temperature increases. This paper proposes an explanation for temperature increase utilizing advanced statistical models such as ARMAX and neural networks with explainable AI components aiming to quantify the effects of these land use changes on local temperature trends over several years. For this purpose, two cities were chosen to apply the methodology. São Félix do Xingu, in the Brazilian Amazon rainforest (a city that has experienced significant deforestation due to agriculture over the last decade), and Cajazeiras, in the Brazilian semi-arid region (a city that has had substantial urban development over the years). The Data for this analysis were sourced from the MapBiomas Brazilian database and temperature records collected by the Brazilian National Institute of Meteorology. Our approach offers insights into human activities’ climatic impact, giving weight to soil changes and providing a methodology for further interdisciplinary studies, potentially serving as a starting point for specialist systems across various domains.
Gender-Neutral English to Portuguese Machine Translator: Promoting Inclusive Language
Machine translation (MT) plays a crucial role in globalization, making access to information more inclusive, although challenges persist for less popular languages, like Portuguese. One of the most complex challenges in the automatic translation into languages such as Portuguese is the precise preservation of masculine and feminine grammatical gender. There are still situations where translation does not adequately reflect gender equality, often reinforcing societal stereotypes. We aim to explore approaches to ensure fairness in English to Portuguese MT through post-processing techniques, which aim to apply some transformation to the model’s output. To this end, we used the MarianMT model as our foundation, then we fine-tuned it using a dataset of English-Portuguese sentences that was generated and carefully crafted to mitigate gender bias within the sentences. The results on gender disparities metrics, based on the WinoMT test set for MT such as ΔG, ΔS, and the overall accuracy (preserving the gender of the entity from the original) significantly improved with some drop in BLEU (Bilingual Evaluation Understudy) score. Our study focuses on addressing gender bias in the Portuguese language. However, it can also be adapted to other languages, since it is crucial to ensure truly fair and inclusive global communication.
Hybrid Artificial Intelligence Model for Detecting Signs of Delayed Child Development
Child development Assessment is a multifaceted process that incorporates variables of diverse origins in order to identify developmental delays. The present study proposes a hybrid artificial intelligence model, combining first-order logic and fuzzy logic to identify delays in child development. The usage of first-order logic facilitates the integration of large volumes of data, promoting a holistic view. The usage of fuzzy logic enables the treatment of uncertainties and a detailed analysis of variables. The results indicate that the proposed model is effective in mapping delays in child development, as well as in using the data obtained to map the child’s evolution trend.
Flávia Luciana Guimaraes Marçal Pantoja de Araújo
Low Birth Weight in Brazil Vulnerable Groups: An Analysis Based on Data Mining and Big Data
Low Birth Weight (LBW) is the most important isolated risk factor for infant mortality. Several studies are conducted to understand the features related to the increased of LBW rate in Brazil. However, few works aggregate national and special vulnerable groups analysis. Furthermore, some data mining methodologies with potential to discover new information from datasets, such as subgroup discovery, have not yet been applied for this purpose. Thus, this study aims to identify and analysis features sets related to LBW in low-income families from Brazil and comparing them with six more vulnerable groups: small towns, rural areas, illiterate mothers, adolescent mothers, indigenous and quilombolas. We also development a dashboard for publicize features set related to LBW in several Brazilian cities.
Modeling EEG Data into Graphs for the Prognostic of Patients in Coma Using Graph Neural Networks
In this work we present a modeling approach using Graph Neural Networks (GNNs) to support the prognosis of patients in coma (PPC) based on Electroencephalogram (EEG) exams. Coma is a critical medical condition characterized by an extended state of unconsciousness. EEG is an exam that captures the electrical activity of the brain through electrodes positioned on the patient’s scalp. It provides valuable information about cerebral functions and can also be used for the monitoring and diagnosis of several diseases. In this study, we investigate four GNN architectures specifically designed for PPC analysis, leveraging the power of convolutional-based layers. In addition, we also develop a modeling strategy to represent EEG sequential data into a graph, named EEGraph, effectively harnessing the spatial positions of electrodes. The main contribution of representing the EEG exam in a graph structure is also to explore all the inherent spatial information the exam has. As it is a highly sensitive exam with generally noisy results on the micro-volt scale, its spatial exploration may reveal the potential to improve the PPC task. Experimental results considering real data showed that our proposed architecture outperformed the LSTM model in terms of F1-score and accuracy. Statistical tests evidenced the enhanced predictive performance achieved by exploiting the spatial aspects of EEG data using GNNs. Moreover, the findings highlight the significant potential of GNNs to contribute to the PPC task from EEG data.
Tackling Low-Resource ECG Classification with Self-supervised Learning
Cardiovascular disease is one of the leading causes of worldwide death and an annual burden for European and American economies. The primary source of data to identify heart abnormalities is the electrocardiogram (ECG). However, its identification requires significant human efforts. In many cases, traditional rule-based diagnosis is inefficient due to the data heterogeneity, motivating the use of deep neural network approaches. However, ECG data varies significantly between equipment, including the sampling rate. Self-supervised Representation Learning (SSL) has gained increasing attention due to its generalization capacity in scenarios with scarce availability of a large volume of labeled data. In this work, we investigate using SSL methods in a cross-dataset scenario. We raised different research questions supported by the assumption that high-resource data, with a large sampling rate and volume of records, is helpful for training models in an SSL manner to classify ECG in low-resource datasets. Besides, we evaluate different resampling methods to match the sampling rate between datasets, including a novel upsampler based on ESPCN. Our experimental evaluation shows that using the state-of-the-art SSL models TS-TCC and TS2Vec can improve the accuracy in the low-resource dataset. However, the best results were obtained using the same dataset in both pre-training and downstream task steps. These results indicate the need to develop more suitable pretext tasks for cross-dataset scenarios, which may use our experimental analysis to build new ideas.
Tuning Hypothesis Creation: Combining Discrete and Continuous Spaces for Zero-Shot Hate Speech Detection
Hate speech detection is a crucial endeavor in maintaining the safety of online spaces, but the effectiveness of supervised approaches hinges mainly on the availability of annotated data. Prior research has explored the utility of natural language inference (NLI) models for zero-shot hate speech detection (ZSHSD), which leverages the capacity of these models to learn semantic relationships and adapt to downstream task without relying on large annotated datasets. NLI models assess if a premise sentence logically entails a hypothesis sentence, relying on precise hypothesis design to achieve adequate downstream task performance. Existing frameworks that use NLI model for ZSHSD rely on multiple inferences with different hypothesis to extract characteristics to achieve desirable outcomes. In light of the challenges surrounding ZSHSD and the method of relying on discrete tokens to design hypotheses, we aim to optimize and identify ideal intermediate representations by applying p-tuning techniques. On HateCheck, a fully tuned hypothesis led to a 18.8% point (pp) accuracy improvement, compared to a discrete designed hypothesis. Our work surpassed prior work by achieving a 5.6 pp accuracy enhancement, outperforming previous approaches that required multiple inferences. Also, the optimized tokens uncover relations to broader aspects of hate speech, offering insights for hypothesis design.