A sketch for the KS test for Big Data - Galeno, Thalis D.; Gama, João; Cardoso, Douglas O. (Anais do Symposium on Knowledge Discovery, Mining and Learning (KDMiLe); 2021: Anais do IX Symposium on Knowledge Discovery, Mining and LearningDO - 10.5753/kdmile.2021.17455, 2021)
Motivated by the challenges of Big Data, this paper presents an approximative algorithm to assess the Kolmogorov-Smirnov test. This goodness of fit statistical test is extensively used because it is non-parametric. This work focuses on the one-sample test, which considers the hypothesis that a given univariate sample follows some reference distribution. The method allows to evaluate the departure from such a distribution of a input stream, being space and time efficient. We show the accuracy of our algorithm by making several experiments in different scenarios: varying reference distribution and its parameters, sample size, and available memory. The performance of rival methods, some of which are considered the state-of-the-art, were compared. It is demonstrated that our algorithm is superior in most of the cases, considering the absolute error of the test statistic.