Detecting the presence of meteors in images: new collection and results

• Renato Moraes Silva UFSCAR
• Ana Carolina Lorena UNIFESP
• Tiago A. Almeida UNIFESP

DOI: https://doi.org/10.5753/eniac.2018.4410

Resumo

In this paper, we present a new public and real dataset of labeled images of meteors and non-meteors that we recently used in a machine learning competition. We also present a comprehensive performance evaluation of several established machine learning methods and compare the results with a stacking approach – one of the winning solutions of the competition. We compared the performance obtained by the methods in the traditional repeated five-fold cross-validation with the ones obtained using the training and test partitions used in the competition. A careful analysis of the results indicates that, in general, the stacking based approach obtained the best performances compared to the baselines. Moreover, we found evidence that the validation strategy used by the platform that hosted the competition can lead to results that do not sustain in a cross-validation setup, which is recommendable in real-world scenarios.

Referências

Abo-Zaid, A., Hinton, O. R., and Horne, E. (1988). About moment normalization and complex moment descriptors. In Pattern Recognition, pages 399–409. Springer.

Bosch, A., Zisserman, A., and Munoz, X. (2007). Representing shape with a spatial pyramid kernel. In Proceedings of the 6th ACM international conference on Image and video retrieval, pages 401–408. ACM.

Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (COLT’92), pages 144–152, Pittsburgh, PA, USA. ACM.

Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2):123–140.

Breiman, L. (2001). Random forests. Machine Learning, 45(1):5–32.

Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984). Classification and Regression Trees. Wadsworth International Group, Belmont, California, USA. Chatzichristofis, S. A. and Boutalis, Y. S. (2008a). Cedd: color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. In International Conference on Computer Vision Systems, pages 312–322. Springer.

Chatzichristofis, S. A. and Boutalis, Y. S. (2008b). Fcth: Fuzzy color and texture histogram-a low level feature for accurate image retrieval. In Image Analysis for Multimedia Interactive Services, 2008. WIAMIS’08. Ninth InternationalWorkshop on, pages 191–196.

IEEE. Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16(1):321–357.

Cortes, C. and Vapnik, V. N. (1995). Support-vector networks. Machine Learning, 20(3):273–297.

Cover, T. M. and Hart, P. E. (1967). Nearest neighbor pattern classification. IEEE Transaction on Information Theory, 13(1):21–27.

Fernández, A., García, S., del Jesus, M. J., and Herrera, F. (2008). A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets. Fuzzy Sets and Systems, 159(18):2378–2398.

Ferri, C., Hernández-Orallo, J., and Modroiu, R. (2009). An experimental comparison of performance measures for classification. Pattern Recognition Letters, 30(1):27–38.

Fogel, I. and Sagi, D. (1989). Gabor filters as texture discriminator. Biological cybernetics, 61(2):103–113.

Freund, Y. and Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139.

Gural, P. S. (2012). A new method of meteor trajectory determination applied to multiple unsynchronized video cameras. Meteoritics & Planetary Science, 47(9):1405–1418.

Han, J. and Ma, K.-K. (2002). Fuzzy color histogram and its use in color image retrieval. IEEE transactions on image processing, 11(8):944–952.

Haralick, R. M., Shanmugam, K., et al. (1973). Textural features for image classification. IEEE Transactions on systems, man, and cybernetics, (6):610–621.

Huang, J., Kumar, S. R., Mitra, M., Zhu, W.-J., and Zabih, R. (1997). Image indexing using color correlograms. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, pages 762–768. IEEE.

Kriegel, H.-P., Schubert, E., and Zimek, A. (2011). Evaluation of multiple clustering solutions. In MultiClust@ ECML/PKDD, pages 55–66.

Metsis, V., Androutsopoulos, I., and Paliouras, G. (2006). Spam filtering with naive Bayes – which naive Bayes? In Proceedings of the 3rd Conference on Email and Anti-Spam (CEAS’06), pages 27–28, Mountain View, California.

Novak, C. L. and Shafer, S. A. (1992). Anatomy of a color histogram. In Computer Vision and Pattern Recognition, 1992. Proceedings CVPR’92., 1992 IEEE Computer Society Conference on, pages 599–605.

IEEE. Ojala, T., Pietikainen, M., and Harwood, D. (1994). Performance evaluation of texture measures with classification based on kullback discrimination of distributions. In Pattern Recognition, 1994. Vol. 1-Conference A: Computer Vision & Image Processing., Proceedings of the 12th IAPR International Conference on, volume 1, pages 582–585.

IEEE. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.

Roman, V. ¸ S. and Buiu, C. (2015). Automatic detection of meteors in spectrograms using artificial neural networks. In Applied Computational Intelligence and Informatics (SACI), 2015 IEEE 10th Jubilee International Symposium on, pages 131–134. IEEE.

Scott, D. W. (2010). Averaged shifted histogram. Wiley Interdisciplinary Reviews: Computational Statistics, 2(2):160–164.

Sikora, T. (2001). The mpeg-7 visual standard for content description-an overview. IEEE Transactions on circuits and systems for video technology, 11(6):696–702.

Silad¯i, E., Vida, D., and Nyarko, E. K. (2015). Video meteor detection filtering using soft computing methods. In Proceedings of the International Meteor Conference, Mistelbach, Austria, pages 27–30.

Sokolova, M. and Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4):427–437.

Tamura, H., Mori, S., and Yamawaki, T. (1978). Textural features corresponding to visual perception. IEEE Transactions on Systems, man, and cybernetics, 8(6):460–473.

Taylor, S. and Drummond, T. (2011). Binary histogrammed intensity patches for efficient and robust matching. International journal of computer vision, 94(2):241–265.

Van De Sande, K., Gevers, T., and Snoek, C. (2010). Evaluating color descriptors for object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 32(9):1582–1596.

van de Sande, K. E., Gevers, T., and Snoek, C. G. (2004). Evaluation of color descriptors for object and scene recognition. In IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, USA (June 2008).

Vítek, S. and Nasyrova, M. (2018). Real-time detection of sporadic meteors in the intensified tv imaging systems. Sensors, 18(1):77. Yu, H.-F., Huang, F.-L., and Lin, C.-J. (2011). Dual coordinate descent methods for logistic regression and maximum entropy models. Machine Learning, 85(1-2):41–75.

Zagoris, K., Chatzichristofis, S. A., Papamarkos, N., and Boutalis, Y. S. (2010). Automatic image annotation and retrieval using the joint composite descriptor. In Informatics (PCI), 2010 14th Panhellenic Conference on, pages 143–147. IEEE.