Experimental Evaluation of Machine Learning Algorithms for Classifying Health-Related News with Indications of Irregularity - Guimarães, Alysson; Colaço Junior, Methanias; Almeida, Samuel; Fontes, Raphael; Prado, Helder (Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI), 2026)
Research Context: The increasing production of unstructured data, particularly textual data, has driven the application of Natural Language Processing (NLP) techniques in public administration. The analysis of multiple information sources enables the identification of patterns and the development of predictive models to optimize strategies, improve service delivery, and ensure population monitoring and safety. Scientific and/or Practical Problem: The auditing process is characterized by high costs, long duration, and heavy reliance on human and material resources, necessitating solutions capable of automating the analysis of corruption reports. Proposed Solution and/or Analysis: Focusing on the preliminary identification of potential irregularities, the use of machine learning models is proposed to support the auditing process by identifying health-related news that may indicate irregularities. Related IS Theory: This study is grounded in Cognitive Load Theory, as it examines methods to reduce information overload. Research Method: A controlled in vitro experiment was conducted to scientifically evaluate 54 machine learning models and compare metrics including Accuracy, Precision, Recall, and F1-score. Additionally, an asymptotic complexity analysis of the algorithms was performed. Summary of Results: The Random Forest model stood out in terms of effectiveness, achieving an accuracy of 99.90%, a recall of 98.62%, and an F1-score of 99.28%, while Naive Bayes and Logistic Regression excelled in efficiency, with linear complexity 𝑂 (𝑛𝑑) for both training and prediction and low memory usage. Contributions and Impact to IS area: The results demonstrate the feasibility of using machine learning models to identify health-related news with potential irregularities. This approach enhances the information gathering and corruption evidence stage performed by AudSUS auditors for detecting potential irregularities, thereby contributing to the efficiency of public resource management.