HSAE: A Hybrid Unsupervised Autoencoder for Zero-Day Attack Detection in Realistic Environments - Silva, Fabiano C. da; Venceslau, Franklin A. M.; Souza, Rafael R. de; Monteiro, José A. S. (Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI); 2026: Anais do XXII Simpósio Brasileiro de Sistemas de Informação, 2026)
Research Context: Anomaly detection in computer networks is essential for cybersecurity, especially in complex environments like corporate and IoT networks that face emerging threats. Scientific and/or Practical Problem: Traditional Intrusion Detection Systems (IDS) struggle to identify zero-day attacks, creating significant security gaps. Signature-based approaches fail against unknown threats, while supervised models require large volumes of labeled data rarely available in practice. Unsupervised methods often produce high false positive rates (FPR), limiting their operational viability. Proposed Solution and/or Analysis: This work proposes HSAE (Hybrid Scoring Autoencoder), a lightweight hybrid deep learning model for network anomaly detection. The solution combines autoencoder reconstruction error with an auxiliary output trained using pseudo-labels, enabling the model to learn normality patterns exclusively from benign traffic. Related IS Theory: The research is grounded in anomaly detection theory via machine learning, integrating unsupervised approaches. The model uses Autoencoder (AE) architecture and its principles of data compression and reconstruction to identify behavioral deviations. Research Method: The study conducted quantitative and empirical evaluation, training and testing the model on public datasets CICIDS2017 and ToN_IoT. HSAE performance was compared to Variational Autoencoder (VAE), the state-of-the-art, using metrics including AUC, precision, recall, F1-Score, and FPR. Summary of Results: HSAE consistently outperformed VAE, achieving AUC of 0.94 in corporate scenarios and 0.86 in IoT ransomware detection. Results demonstrated the model’s utility as a practical tool for high-precision anomaly identification with significantly lower false positive rates. Contributions and Impact to IS area: This work contributes to Information Systems by presenting an anomaly detection model tailored for realistic scenarios, offering practical approaches to managing zero-day threats. Its computational efficiency enables implementation in resource-constrained environments like IoT and industrial systems where security is critical.