Application of Hashing Techniques and Convolutional Neural Networks in Reverse Image Search

• Sandy Hoffmann Federal Institute of Santa Catarina (IFC) http://orcid.org/0009-0004-7471-8784
• Ricardo de la Rocha Ladeira Federal Institute of Santa Catarina (IFC) https://orcid.org/0000-0002-3347-083X

DOI: https://doi.org/10.5753/errc.2024.4680

Abstract

This paper explores techniques used in reverse image search, focusing on perceptual hashing and convolutional neural networks (CNNs), aiming to compare their accuracy rates. The experiments involved manipulating images with various effects including Gaussian blur, desaturation (grayscale), resolution reduction, sharpness enhancement, 90° rotation, mirroring, cropping, and the addition of graphic element. Perceptual hashing generates hash values that exhibit minimal changes when the input undergoes slight modifications. The paper examines a variation of perceptual hashing using the discrete cosine transform (DCT). CNN, a type of artificial intelligence typically applied to images, use convolution operations to extract specific features from data. These features can be used for reverse image search. The experiments reveal that CNNs are more resilient to alterations such as sharpness enhancement, cropping, rotation, and mirroring. Additionally, this paper discusses a solution involving random locality-sensitive hashing (LSH), a partitioning technique that uses random hyperplanes to divide the search space. The combination of random LSH and CNNs is presented as a solution to enhance search accuracy. The experiments demonstrating improved accuracy rates for images altered by Gaussian blur, grayscale, and cropping. In conclusion, compared to the DCT, the use of feature vectors extracted from the layers of a CNN constitutes a more robust alternative, and the application of LSH to reduce the search space effectively preserves the accuracy rate.

Keywords: Reverse Image Search, Perceptual Hashing, Convolutional Neural Network, Locality Sensitive Hashing

References

Wan, G. G.; Liu, Z. Content-based information retrieval and digital libraries. Information Technology and Libraries, 2008. ISSN 0730-9295, 2163-5226. DOI: 10.6017/ital.v27i1.3262. Disponível em: [link].

Wekesa, E. N.; DeCusatis, C.; Zhu, A. A Black Box Comparison of Machine Learning Reverse Image Search for Cybersecurity OSINT Applications. Electronics, v. 12, n. 23, 2023. ISSN 2079-9292. DOI: 10.3390/electronics12.234822. Disponível em: [link].

Asian and Pacific Training Centre for ICT for Development (APCICT). Course on Trust and Security Using Digital Technologies. Institutional Repository - ESCAP, 2024. Web. Disponível em: [link].

George, A. S.; Baskar, D.; Pandey, D. Establishing Global AI Accountability: Training Data Transparency, Copyright, and Misinformation. Partners Universal Innovative Research Publication (PUIRP), v. 02, n. 03, jun. 2024. DOI: 10.5281/zenodo.11659602.

Qazanfari, H.; AlyanNezhadi, M. M.; Khoshdaregi, Z. N. Advancements in Content-Based Image Retrieval: A Comprehensive Survey of Relevance Feedback Techniques. 2023. arXiv: 2312. 10089 [cs.CV]. Disponível em: [link].

Gaillard, M. Perceptual Hashing using Convolutional Neural Networks for Large Scale Reverse Image Search. INSA, 2017. Disponível em: [link].

Piosenka, G. Butterfly and Moths Image Classification 100 species. 2022. Accessed: 2024-10-04. Disponível em: [link].

Farid, H. An Overview of Perceptual Hashing. Journal of Online Trust e Safety, 2021. Disponível em: [link].

Khelifi, F.; Bouridane, A. Perceptual Video Hashing for Content Identification and Authentication. IEEE Transactions on Circuits and Systems for Video Technology, v. 29, n. 1, p. 50–67, 2019. DOI: 10.1109/TCSVT.2017.2776159.

Rajaram, S.; Scholz, M. Client-friendly classification over random hyperplane hashes. In: SPRINGER. JOINT European Conference on Machine Learning and Knowledge Discovery in Databases. 2008. P. 250–265.

Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning. MIT Press, 2016. [link].

Du, L.; Ho, A. T.; Cong, R. Perceptual hashing for image authentication: A survey. ELSEVIER, 2020. DOI: 10.1016/j.image.2019.115713.

Zauner, C. Implementation and Benchmarking of Perceptual Image Hash Functions. v. 43. 2010. Disponível em: [link].

Weidman, S. Deep Learning from Scratch: Building with Python from First Principles. O’Reilly Media, Incorporated, 2019. ISBN 9781492041412. Disponível em: [link].

Jafari, O. et al. A Survey on Locality Sensitive Hashing Algorithms and their Applications. CoRR, abs/2102.08942, 2021. arXiv: 2102.08942. Disponível em: [link].

Lavrenko, V. LSH.9 Locality-sensitive hashing: how it works. 2015. YouTube. Accessed: 2024-10-07. Disponível em: [link].

Alomar, K.; Aysel, H. I.; Cai, X. Data Augmentation in Classification and Segmentation: A Survey and New Strategies. Journal of Imaging, v. 9, n. 2, 2023. ISSN 2313-433X. DOI: 10.3390/jimaging9020046. Disponível em: [link].

Samanta, P.; Jain, S. Analysis of Perceptual Hashing Algorithms in Image Manipulation Detection. Procedia Computer Science, v. 185, p. 203–212, 2021. Big Data, IoT, and AI for a Smarter Future. ISSN 1877-0509. DOI: 10.1016/j.procs.2021.05.021. Disponível em: [link].

Schiavo, A. et al. Sketches image analysis: Web image search engine usingLSH index and DNN InceptionV3. 2021. arXiv: 2105. 01147 [cs.CV]. Disponível em: [link].