WOCI: Weighted Operational Cost Index A New Evaluation Metric for Intrusion Detection

• Vagner E. Quincozes UFF
• Silvio E. Quincozes UNIPAMPA
• Célio Albuquerque UFF
• Diego Passos ISEL
• Daniel Mossé PITT

DOI: https://doi.org/10.5753/sbseg.2026.29109

Resumo

This paper proposes the Weighted Operational Cost Index (WOCI), a post-hoc metric for evaluating binary detection configurations under asymmetric error costs. WOCI combines averaged counts of false positives and false negatives through a configurable cost ratio R = CFN/CFP , normalized by the number of positive instances. We instantiate WOCI in an Intrusion Detection System (IDS) case study using the ERENO and 5G-NIDD datasets with CatBoost and XGBoost, comparing configurations from Built-in importance, SHAP, and Mutual Information across multiple feature subset sizes. Results show that 13 of 20 configurations keep the same preferred strategy across R ∈ [1, 100], while seven change at a crossover point R^∗. The main transitions occur in 5G-NIDD with XGBoost near R ≈ 10, a plausible enterprise cost scenario.

Referências

ANEEL (2025). Estabelecida a receita anual permitida (RAP) de transmissoras para o ciclo 2025–2026. [link]. Accessed: 2026-04-27.

ANEEL (2026). Receita anual permitida de transmissão. [link]. Accessed: 2026-04-27.

Cha, J., Jang, J., Shin, D., and Shin, D. (2026). Cost-sensitive threshold optimization for network intrusion detection: A per-class approach with xgboost. Electronics, 15(7):1542.

Chen, Z., Xia, W., Li, Z., Xiong, G., and Gou, G. (2024). RecoSelector: Cost-Sensitive Feature Selection for Network Intrusion Detection in Resource-Constrained Internet of Things. In International Performance, Computing, and Communications Conference, pages 1–10. IEEE.

Drummond, C. and Holte, R. C. (2006). Cost curves: An improved method for visualizing classifier performance. Machine learning, 65(1):95–130.

Elkan, C. (2001). The Foundations of Cost-Sensitive Learning. In International Joint Conference on Artificial Intelligence, volume 17, pages 973–978.

Fatima, M., Rehman, O., Rahman, I. M., Ajmal, A., and Park, S. J. (2024). Towards ensemble feature selection for lightweight intrusion detection in resource-constrained IoT devices. Future Internet, 16(10):368.

Gombar, M., Topalović, A., and Pejić Bach, M. (2026). Cost-Aware Lightweight Deep Learning for Intrusion Detection: A Comparative Study on UNSW-NB15 and CICIDS2017. Electronics, 15(8):1603.

Gupta, N., Jindal, V., and Bedi, P. (2022). CSE-IDS: Using cost-sensitive deep learning and ensemble algorithms to handle class imbalance in network-based intrusion detection systems. Computers & Security, 112:102499.

Hozouri, A., Mirzaei, A., and Effatparvar, M. (2025). A comprehensive survey on intrusion detection systems with advances in machine learning, deep learning and emerging cybersecurity challenges. Discover Artificial Intelligence, 5(1):314.

IBM (2025). Cost of a data breach report 2025. [link]. Accessed: 2026-04-27.

Lee, W., Fan, W., Miller, M., Stolfo, S. J., and Zadok, E. (2002). Toward cost-sensitive modeling for intrusion detection and response. Journal of computer security, 10(1-2):5–22.

Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.

Nelson, A., Rekhi, S., Souppaya, M., and Scarfone, K. (2025). Incident Response Recommendations and Considerations for Cybersecurity Risk Management: A CSF 2.0 Community Profile. Technical Report NIST Special Publication 800-61 Revision 3, National Institute of Standards and Technology.

Operador Nacional do Sistema Elétrico (2019). Submódulo 15.12: Apuração mensal das parcelas variáveis referentes à disponibilidade de instalações da Rede Básica e das Interligações Internacionais. ons.org.br/procedimentos-de-rede/subm odulo-15.12. Accessed: 2026-04-27.

Papaioannou, N., Myllis, G., Tsimpiris, A., and Vrana, V. (2025). The Role of Mutual Information Estimator Choice in Feature Selection: An Empirical Study on mRMR. Information, 16(9):724.

Quincozes, S. E., Albuquerque, C., Passos, D., and Mossé, D. (2023). ERENO: A Framework for Generating Realistic IEC–61850 Intrusion Detection Datasets for Smart Grids. IEEE Transactions on Dependable and Secure Computing, 21(4):3851–3865.

Siriwardhana, Y., Samarakoon, S., Porambage, P., Liyanage, M., Chang, S.-Y., Kim, J., Kim, J., and Ylianttila, M. (2025). Descriptor: 5G Wireless Network Intrusion Detection Dataset (5G-NIDD). IEEE Data Descriptions.

Skrodelis, H. K. and Romanovs, A. (2026). Explainable Hybrid Intrusion Detection for SCADA/ICS: A Review and Research Agenda. Frontiers in Computer Science.

Stolfo, J., Fan, W., Lee, W., Prodromidis, A., and Chan, P. K. (2000). Cost-based modeling and evaluation for data mining with application to fraud and intrusion detection. Results from the JAM Project by Salvatore, pages 1–15.

Telikani, A. and Gandomi, A. H. (2021). Cost-sensitive stacked auto-encoders for intrusion detection in the internet of things. Internet of Things, 14:100122.

U.S. Bureau of Labor Statistics (2025). Information Security Analysts: Occupational Outlook Handbook. [link]. Accessed: 2026-04-27.

Wang, H., Liang, Q., Hancock, J. T., and Khoshgoftaar, T. M. (2024). Feature selection strategies: a comparative analysis of shap-value and importance-based methods. Journal of Big Data, 11(1):44.

Westphal, C., Hailes, S., and Musolesi, M. (2025). Feature selection for network intrusion detection. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD ’25, page 1599–1610, New York, NY, USA.