A strategy for developing a pair of diverse deep learning classifiers for use in a 2-classifier system
Resumo
A multi-classifier system requires the individual classifiers to be both accurate and diverse. There is always a trade-off between accuracy and diversity. One of the suitable ways to enforce diversity between the classifiers is to use different neural network architectures, hyperparameters and training dataset. However, in general, these individual classifiers are trained by mapping the label into a binary vector such that all the incorrect classes are penalized equally. In this paper, we propose a methodology of generating the label vector into two different forms by exploiting the similarity between the classes. One of these forms of label vector encode values such that the misclassification into a class of lower similarity is penalized more than the misclassification into a class of higher similarity, whereas the second form of label vector encode values to do the opposite. These two forms of label vectors are then used for training two individual classifiers. The results show that the pair of classifiers developed with this proposed approach, when used in a 2-classifier system, yields better performance in terms of both diversity and accuracy as compared to the pair of classifiers trained with the standard approach of penalizing all the incorrect classes equally via binary label vector.
Referências
R. Socher B. Huval B. Bath C. D. Manning and A. Y. Ng "Convolutional-recursive deep learning for 3d object classification" Advances in neural information processing systems pp. 656-664 2012.
D. Ciregan U. Meier and J. Schmidhuber "Multi-column deep neural networks for image classification" 2012 IEEE Conference on Computer Vision and Pattern Recognition pp. 3642-3649 2012.
C. Chen A. Seff A. Kornhauser and J. Xiao "Deepdriving: Learning affordance for direct perception in autonomous driving" 2015 IEEE International Conference on Computer Vision (ICCV) pp. 2722-2730 2015.
B. Huval et al. An empirical evaluation of deep learning on highway driving 2015.
A. E. Sallab M. Abdou E. Perot and S. Yogamani "Deep reinforcement learning framework for autonomous driving" Electronic Imaging vol. 2017 no. 19 pp. 70-76 2017.
V. Gao F. Turek and M. Vitaterna "Multiple classifier systems for automatic sleep scoring in mice" Journal of neuroscience methods vol. 264 pp. 33-39 2016.
D. Shi and X. Yang "Mapping vegetation and land cover in a large urban area using a multiple classifier system" International Journal of Remote Sensing vol. 38 no. 16 pp. 4700-4721 2017.
S. Barak A. Arjmand and S. Ortobelli "Fusion of multiple diverse predictors in stock market" Information Fusion vol. 36 pp. 90-102 2017.
M. Cisse P. Bojanowski E. Grave Y. Dauphin and N. Usunier "Parseval networks: Improving robustness to adversarial examples" 34th International Conference on Machine Learning (ICML) 2017.
A. Madry A. Makelov L. Schmidt D. Tsipras and A. Vladu Towards deep learning models resistant to adversarial attacks 2017.
C. Agarwal A. Nguyen and D. Schonfeld "Improving robustness to adversarial examples by encouraging discriminative features" 2019 IEEE International Conference on Image Processing (ICIP) pp. 3801-3505 2019.
L. Metz N. Maheswaranathan J. Shlens J. Sohl-Dickstein and E. D. Cubuk Using learned optimizers to make models robust to input noise 2019.
S. Schneider E. Rusak L. Eck O. Bringmann W. Brendel and M. Bethge Improving robustness against common corruptions by covariate shift adaptation 2020.
F. Machida "N-version machine learning models for safety critical systems" 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W) pp. 48-51 2019.
H. K. Butler M. A. Friend K. W. Bauer and T. J. Bihl "The effectiveness of using diversity to select multiple classifier systems with varying classification thresholds" Journal of Algorithms & Computational Technology vol. 12 no. 3 pp. 187-199 2018.
Y. Yang D. Han and J. Dezert "A ranking distance based diversity measure for multiple classifier systems" 2018 International Conference on Control Automation and Information Sciences (ICCAIS) pp. 55-60 2018.
G. Huang Y. Li G. Pleiss Z. Liu J. E. Hopcroft and K. Q. Weinberger Snapshot ensembles: Train 1 get m for free 2017.
A. Wasay B. Hentschel Y. Liao S. Chen and S. Idreos "Mothernets: Rapid deep ensemble learning" Proceedings of the 3rd MLSys Conference (MLSys) 2020.
P. S. Negi M. Mahoor et al. Leveraging class similarity to improve deep neural network robustness 2018.
S. Sengupta A. Dudley T. Chakraborti and S. Kambhampati "An investigation of bounded misclassification for operational security of deep neural networks" AAAI Workshop of Engineering Dependable and Secure Maching Learning Systems 2018.
R. Kashyap "The perfect marriage and much more: Combining dimension reduction distance measures and covariance" Physica A: Statistical Mechanics and its Applications vol. 536 2019.
S. Kullback and R. A. Leibler "On information and sufficiency" The annals of mathematical statistics vol. 22 no. 1 pp. 79-86 1951.
J. Duchi "Derivations for linear algebra and optimization" Berkeley California vol. 3 pp. 2325-5870 2007.
A. Subramanya S. Srinivas and R. V. Babu Confidence estimation in deep neural networks via density modelling 2017.
A. Kendall and Y. Gal "What uncertainties do we need in bayesian deep learning for computer vision?" Advances in neural information processing systems pp. 5574-5584 2017.
J. Stallkamp M. Schlipsing J. Salmen and C. Igel "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition" Neural Networks vol. 32 pp. 323-332 2012.
