The impact of image transformations in the context of self-supervised learning approaches using contrastive learning

• Misael S. de Rezende UFMG
• Jesimon Barreto UFMG
• William R. Schwartz UFMG

DOI: https://doi.org/10.5753/sibgrapi.est.2024.31658

Abstract

This research investigates the impact of image transformations in the context of learning self-supervised, especially when combined with contrastive learning techniques. Our objective is evaluate how various image transformations influence the quality of the learned representations and, consequently, the overall performance of the model. By focusing on the limitations of existing methods, including the LEWEL model, our study seeks to deepen the understanding of the effects of transformations of images in self-supervised learning. Across experiments on the ImageNet-100 dataset, we explored the implications of transformations in representations and their transferability to linear classification.

References

A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive self-supervised learning,” 2021.

J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko, “Bootstrap your own latent: A new approach to self-supervised learning,” 2020.

K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” 2020.

L. Huang, S. You, M. Zheng, F. Wang, C. Qian, and T. Yamasaki, “Learning where to learn in cross-view self-supervised learning,” 2022.

T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” 2020.

X. Chen, H. Fan, R. Girshick, and K. He, “Improved baselines with momentum contrastive learning,” 2020.

M. C. Schiappa, Y. S. Rawat, and M. Shah, “Self-supervised learning for videos: A survey,” ACM Computing Surveys, dec 2022. [Online]. Available: DOI: 10.1145\%2F3577925

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, no. 2, pp. 336–359, oct 2019. [Online]. Available: DOI: 10.1007%2Fs11263-019-01228-7

Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” 2020.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “Imagenet large scale visual recognition challenge,” 2015.