Using instance hardness measures in curriculum learning

• Gustavo H. Nunes UNIFESP
• Gustavo O. Martins ITA
• Carlos H. Q. Forster ITA
• Ana C. Lorena ITA

DOI: https://doi.org/10.5753/eniac.2021.18251

Resumo

Curriculum learning consists of training strategies for machine learning techniques in which the easiest observations are presented first, progressing into more difficult cases as training proceeds. For assembling the curriculum, it is necessary to order the observations a dataset has according to their difficulty. This work investigates how instance hardness measures, which can be used to assess the difficulty level of each observation in a dataset from different perspectives, can be used to assemble a curriculum. Experiments with four CIFAR-100 sub-problems have demonstrated the feasibility of using the instance hardness measures, the main advantage is on convergence speed and some datasets accuracy gains can also be verified.

Referências

Arruda, J. L., Prudêncio, R. B., and Lorena, A. C. (2020). Measuring instance hardness using data complexity measures. In BRACIS 2020, pages 483–497. Springer.

Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009). Curriculum learning. In Proceedings of the 26th ICML, pages 41–48.

Bottou, L. and Lin, C.-J. (2007). Support vector machine solvers. Large scale kernel machines, 3(1):301–320.

Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3):273–297.

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE CVPR, pages 248–255.

Hacohen, G. and Weinshall, D. (2019). On the power of curriculum learning in training deep networks. In ICML, pages 2535–2544. PMLR.

Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images. Citeseer.

Lorena, A. C., Garcia, L. P., Lehmann, J., Souto, M. C., and Ho, T. K. (2019). How complex is your classification problem? a survey on measuring classification complexity. ACM Computing Surveys (CSUR), 52(5):1–34.

Silva, F. L. D. and Costa, A. H. R. (2018). Object-oriented curriculum generation for reinforcement learning. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pages 1026–1034.

Smith, M. R. and Martinez, T. (2016). A comparative evaluation of curriculum learning with filtering and boosting in supervised classification problems. Computational Intelligence, 32(2):167–195.

Smith, M. R., Martinez, T., and Giraud-Carrier, C. (2014). An instance level analysis of data complexity. Machine learning, 95(2):225–256.

Szegedy, C., Vanhoucke, V., Ioe, S., Shlens, J., and Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In IEEE CVPR, pages 2818–2826.

Weinshall, D. and Amir, D. (2020). Theory of curriculum learning, with convex loss functions. Journal of Machine Learning Research, 21(222):1–19.

Weinshall, D., Cohen, G., and Amir, D. (2018). Curriculum learning by transfer learning: Theory and experiments with deep networks. In ICML, pages 5238– 5246. PMLR.