Cutoff Frequency Adjustment for FFT-Based Anomaly Detectors

• Ellen Paixão Silva Centro Federal de Educação Tecnológica Celso Suckow da Fonseca (CEFET/RJ) / Operador Nacional do Sistema Elétrico (ONS)
• Helga Balbi Centro Federal de Educação Tecnológica Celso Suckow da Fonseca (CEFET/RJ)
• Esther Pacitti University of Montpellier / INRIA
• Fabio Porto Laboratório Nacional de Computação Científica (LNCC)
• Joel Santos Centro Federal de Educação Tecnológica Celso Suckow da Fonseca (CEFET/RJ)
• Eduardo Ogasawara Centro Federal de Educação Tecnológica Celso Suckow da Fonseca (CEFET/RJ)

DOI: https://doi.org/10.5753/sbbd.2024.243319

Resumo

This article presents a time series anomaly detection method based on the Fast Fourier Transform (FFT) using a high-pass filter. The proposed method aims to remove low-frequency components, such as trends and seasonality, which represent the normal behavior of the series, while preserving high-frequency components associated with anomalies. The major challenge in constructing this method lies in determining the high-pass filter's cutoff frequency without prior knowledge of the intrinsic nature of the series. In addition to the traditional approach, four new distinct approaches were explored to determine the high-pass filter's cutoff frequency, making the method adaptable to various datasets. Experimental results show the effectiveness of the method in anomaly detection using high-pass FFT filters that have a cutoff frequency adjusted by change points, outperforming traditional techniques such as statistical and machine learning methods in terms of F1 score, precision, accuracy, and execution time.

Palavras-chave: anomaly detection, time series, fourier transform

Referências

Bhattacharya, C., De, S., Mukhopadhyay, A., Sen, S., and Ray, A. (2020). Detection and classification of lean blow-out and thermoacoustic instability in turbulent combustors. Applied Thermal Engineering, 180.

Bürger, F. and Pauli, J. (2013). Unsupervised segmentation of anomalies in sequential data, images and volumetric data using multiscale fourier phase-only analysis. In LNCS, volume 7944, pages 44 – 53.

Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3).

Collins Jackson, A. and Lacey, S. (2020). The discrete Fourier transformation for seasonality and anomaly detection of an application to rare data. Data Technologies and Applications, 54(2):121 – 132.

Erkuş, E. C. and Purutçuoğlu, V. (2021). Outlier detection and quasi-periodicity optimization algorithm: Frequency domain based outlier detection (FOD). European Journal of Operational Research, 291(2):560 – 574.

Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M., and Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4).

Han, J., Pei, J., and Tong, H. (2022). Data Mining: Concepts and Techniques. Morgan Kaufmann, Cambridge, MA, 4th edition edition.

Herrera, M., Proselkov, Y., Perez-Hernandez, M., and Parlikad, A. K. (2021). Mining Graph-Fourier Transform Time Series for Anomaly Detection of Internet Traffic at Core and Metro Networks. IEEE Access, 9:8997 – 9011.

Jiang, J.-R., Kao, J.-B., and Li, Y.-L. (2021). Semi-supervised time series anomaly detection based on statistics and deep learning. Applied Sciences (Switzerland), 11(15).

Killick, R. and Eckley, I. A. (2014). Changepoint: An R package for changepoint analysis. Journal of Statistical Software, 58(3):1 – 19.

Lima, J., Salles, R., Porto, F., Coutinho, R., Alpis, P., Escobar, L., Pacitti, E., and Ogasawara, E. (2022). Forward and Backward Inertial Anomaly Detector: A Novel Time Series Event Detection Method. In Proceedings of the IJCNN, volume 2022-July, pages 1–8.

Lindstrom, M. R., Jung, H., and Larocque, D. (2020). Functional kernel density estimation: Point and fourier approaches to time series anomaly detection. Entropy, 22(12):1 – 15.

Loyarte, M. G. and Menenti, M. (2008). Impact of rainfall anomalies on Fourier parameters of NDVI time series of northwestern Argentina. International Journal of Remote Sensing, 29(4):1125 – 1152.

Lykou, R., Tsaklidis, G., and Papadimitriou, E. (2020). Change point analysis on the Corinth Gulf (Greece) seismicity. Physica A: Statistical Mechanics and its Applications, 541.

Olteanu, M., Rossi, F., and Yger, F. (2023). Meta-survey on outlier and anomaly detection. Neurocomputing, 555.

Oppenheim, A. V., Willsky, A. S., and Nawab, S. H. (1997). Signals & Systems. Prentice Hall. Pang, G., Shen, C., Cao, L., and Van Den Hengel, A. (2021). Deep Learning for Anomaly Detection: A Review. ACM Computing Surveys, 54(2).

Ye, Y., He, Q., Zhang, P., Xiao, J., and Li, Z. (2023). Multivariate Time Series Anomaly Detection with Fourier Time Series Transformer. In 2023 IEEE 12th CloudNet 2023, pages 381 – 388.

Yu, Y., Zhu, Y., Li, S., and Wan, D. (2014). Time series outlier detection based on sliding window prediction. Mathematical Problems in Engineering, 2014.

Zhao, H., Lu, B., Yu, L., Zhao, S., Zeng, L., Zhang, Z., and You, P. (2018). A fourier series-based anomaly extraction approach to access network traffic in power telecommunications. In 2017 ICCSEC, pages 550 – 553.

Zhou, L., Guo, W., Cao, J., Zhang, X., and Wang, Y. (2023). Wavelet-SVDD: Anomaly Detection and Segmentation with Frequency Domain Attention. In LNAI, volume 14177, pages 230 – 243.