DASRS Rest: Um Algoritmo Eficiente de Detecção de Anomalias em Tempo Real para Data Centers
Resumo
A grande quantidade de equipamentos, com diferentes configurações, e atualizações constantes de software e hardware em data centers, tornam difícil o uso de sistemas de monitoração baseados na configuração de thresholds. Este artigo propõe utilizar detecção de anomalias para prevenção de falhas e apresenta o algoritmo Decreased Anomaly Score by Repeated Sequence (DASRS) Rest, que detecta anomalias sem exigir conhecimento prévio do serviço monitorado. Avaliamos o desempenho do DASRS Rest utilizando o framework Numenta Anomaly Benchmark (NAB). O algoritmo proposto possui bons resultados de acurácia, o menor consumo de memória e é o mais rápido, quando comparado com diversos algoritmos do estado da arte.
Palavras-chave:
Detecção de Anomalias, Séries Temporais, Dados de Streaming, Algoritmos, Aprendizagem não supervisionada, Data Center, Métricas, Monitoração
Referências
Adams, R. P. and MacKay, D. J. (2007). Bayesian online changepoint detection. Technical report, University of Cambridge.
Ahmad, S., Lavin, A., Purdy, S., and Agha, Z. (2017). Unsupervised real-time anomaly detection for streaming data. Neurocomputing, 262:134 – 147.
Ahmad, S. and Purdy, S. (2016). Real-Time Anomaly Detection for Streaming Analytics. CoRR, abs/1607.02480.
Brockwell, P. J. and Davis, R. A. (2016). Introduction to time series and forecasting. In Springer Texts in Statistics. Springer International Publishing.
Burnaev, E. and Ishimtsev, V. (2016). Conformalized density-and distance-based anomaly detection in time-series data. arXiv:1608.04585.
Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3):15:1–15:58.
Claps, G. G., Svensson, R. B., and Aurum, A. (2015). On the journey to continuous deployment: Technical and social challenges along the way. Information and Software technology, 57:21–31.
Couto, R. S., Campista, M. E. M., and Costa, L. H. M. (2012). A reliability analysis of datacenter topologies. In IEEE Global Communications Conference (GLOBECOM), pages 1890–1895.
Dias, R. (2019). Detecção de anomalias nas métricas das monitorações de máquinas de um data center. Master’s thesis, Pontifı́cia Universidade Católica do Rio de Janeiro - PUC-Rio.
Earthgecko (2019). Earthgecko skyline. https://github.com/earthgecko/skyline. Acessado em abril de 2019.
Etsy, I. (2013). Skyline anomaly detection system. https://github.com/etsy/skyline. Acessado em abril de 2019.
Fox, A. J. (1972). Outliers in time series. Journal of the Royal Statistical Society. Series B, 34:350–363.
Gabel, M., Schuster, A., Bachrach, R., and Bjørner, N. (2012). Latent fault detection in large scale services. In IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012), pages 1–12.
Guha, S., Mishra, N., Roy, G., and Schrijvers, O. (2016). Robust random cut forest based anomaly detection on streams. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, pages 2712– 2721. JMLR.org.
Kejariwal, A. (2015). Twitter engineering: Introducing practical and robust anomaly detection in a time series. https://bit.ly/2YIPeSA. Acessado em junho de 2019.
Lavin, A. and Ahmad, S. (2015). Evaluating real-time anomaly detection algorithms - the numenta anomaly benchmark. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pages 38–44.
Munir, M., Siddiqui, S., Dengel, A., and Ahmed, S. (2018). Deepant: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access, PP:1–1.
Rosner, B. (1983). Percentage points for a generalized esd many-outlier procedure. Technometrics, 25:165–172.
Safin, A. and Burnaev, E. (2017). Conformal kernel expected similarity for anomaly detection in time-series data. Advances in Systems Science and Applications, 17:22– 33.
Schneider, M., Ertel, W., and Ramos, F. T. (2016). Expected similarity estimation for large-scale batch and streaming anomaly detection. CoRR, abs/1601.06602.
Singh, N. and Olinsky, C. (2017). Demystifying numenta anomaly benchmark. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 1570–1577.
Smirnov, M. (2016). Contextual anomaly detector. https://github.com/smirmik/CAD. Acessado em abril de 2019.
Wang, C., Viswanathan, K., Choudur, L., Talwar, V., Satterfield, W., and Schwan, K. (2011). Statistical techniques for online anomaly detection in data centers. In 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops, pages 385–392.
Ahmad, S., Lavin, A., Purdy, S., and Agha, Z. (2017). Unsupervised real-time anomaly detection for streaming data. Neurocomputing, 262:134 – 147.
Ahmad, S. and Purdy, S. (2016). Real-Time Anomaly Detection for Streaming Analytics. CoRR, abs/1607.02480.
Brockwell, P. J. and Davis, R. A. (2016). Introduction to time series and forecasting. In Springer Texts in Statistics. Springer International Publishing.
Burnaev, E. and Ishimtsev, V. (2016). Conformalized density-and distance-based anomaly detection in time-series data. arXiv:1608.04585.
Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3):15:1–15:58.
Claps, G. G., Svensson, R. B., and Aurum, A. (2015). On the journey to continuous deployment: Technical and social challenges along the way. Information and Software technology, 57:21–31.
Couto, R. S., Campista, M. E. M., and Costa, L. H. M. (2012). A reliability analysis of datacenter topologies. In IEEE Global Communications Conference (GLOBECOM), pages 1890–1895.
Dias, R. (2019). Detecção de anomalias nas métricas das monitorações de máquinas de um data center. Master’s thesis, Pontifı́cia Universidade Católica do Rio de Janeiro - PUC-Rio.
Earthgecko (2019). Earthgecko skyline. https://github.com/earthgecko/skyline. Acessado em abril de 2019.
Etsy, I. (2013). Skyline anomaly detection system. https://github.com/etsy/skyline. Acessado em abril de 2019.
Fox, A. J. (1972). Outliers in time series. Journal of the Royal Statistical Society. Series B, 34:350–363.
Gabel, M., Schuster, A., Bachrach, R., and Bjørner, N. (2012). Latent fault detection in large scale services. In IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012), pages 1–12.
Guha, S., Mishra, N., Roy, G., and Schrijvers, O. (2016). Robust random cut forest based anomaly detection on streams. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, pages 2712– 2721. JMLR.org.
Kejariwal, A. (2015). Twitter engineering: Introducing practical and robust anomaly detection in a time series. https://bit.ly/2YIPeSA. Acessado em junho de 2019.
Lavin, A. and Ahmad, S. (2015). Evaluating real-time anomaly detection algorithms - the numenta anomaly benchmark. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pages 38–44.
Munir, M., Siddiqui, S., Dengel, A., and Ahmed, S. (2018). Deepant: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access, PP:1–1.
Rosner, B. (1983). Percentage points for a generalized esd many-outlier procedure. Technometrics, 25:165–172.
Safin, A. and Burnaev, E. (2017). Conformal kernel expected similarity for anomaly detection in time-series data. Advances in Systems Science and Applications, 17:22– 33.
Schneider, M., Ertel, W., and Ramos, F. T. (2016). Expected similarity estimation for large-scale batch and streaming anomaly detection. CoRR, abs/1601.06602.
Singh, N. and Olinsky, C. (2017). Demystifying numenta anomaly benchmark. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 1570–1577.
Smirnov, M. (2016). Contextual anomaly detector. https://github.com/smirmik/CAD. Acessado em abril de 2019.
Wang, C., Viswanathan, K., Choudur, L., Talwar, V., Satterfield, W., and Schwan, K. (2011). Statistical techniques for online anomaly detection in data centers. In 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops, pages 385–392.
Publicado
07/12/2020
Como Citar
DIAS, Ricardo Souza; MAURICIO, Leopoldo Alexandre Freitas; POGGI DE ARAGÃO, Marcus Vinicius Soledade.
DASRS Rest: Um Algoritmo Eficiente de Detecção de Anomalias em Tempo Real para Data Centers. In: SIMPÓSIO BRASILEIRO DE REDES DE COMPUTADORES E SISTEMAS DISTRIBUÍDOS (SBRC), 38. , 2020, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 672-685.
ISSN 2177-9384.
DOI: https://doi.org/10.5753/sbrc.2020.12317.
