A DCT-Based Contrastive Approach for Learning Visual Representations - Moraes, Daniel de Sousa; Santos, Pedro Cutrim dos; Busson, Antonio José G.; Duarte, Julio Cesar; Colcher, Sérgio (Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia); 2025: Proceedings of the 31st Brazilian Symposium on Multimedia and the WebDO - 10.5753/webmedia.2025.14963, 2025)
Recent advances in self-supervised learning have significantly improved visual representation learning, creating better alternatives to supervised methods in image understanding tasks. Motivated by the demand for robust features under low-quality image scenarios, we propose adapting the SimCLR framework to operate in the frequency domain using quantized DCT coefficients from JPEG-compressed images. We aim to investigate the integration of the Discrete Cosine Transform (DCT) with self-supervised contrastive learning to enable visual representation learning. To evaluate the learned representations, we apply linear classification and JPEG artifact removal tasks, employing ResNet-50 and Vision Transformer (ViT) encoders. Results indicate that ViT-based embeddings, especially when combined with spatial or frequency-domain inputs, mitigate performance loss degraded by compression. Furthermore, artifact removal experiments with modified UNet architectures show that incorporating ViT embeddings and attention maps can slightly improve reconstruction quality. These findings suggest that frequency-domain self-supervised learning is a promising direction for building robust and transferable visual representations. However, the observed gains remain modest and limited by computational constraints.