Through the Eyes of Instagram: Analyzing Image Content utilizing Meta's Automatic Alt-Text

• João Francisco Hecksher Olivetti UFV
• Philipe de Freitas Melo UFV

DOI: https://doi.org/10.5753/webmedia.2024.241695

Resumo

Multimedia communication has become an essential part of social media, with images representing a significant part of the content on most platforms. This study investigates image content on Instagram through Meta’s internal image classification algorithm, Automatic Alt-Text (AAT). Our approach differs from research on data from comments and hashtags because of the use of actual visual descriptions as the means of understanding the kinds of the content published on the network. Our analysis of 200k posts reveals 1,471 unique tags being used to characterize image content on Instagram, representing mostly objects, food, animals, locations and other common components of social media photos. Notably, we found that content about personal aesthetics is highly popular on the platform, with person and selfie being respectively some of the top two most common tag and post categories, being also highly related to other tags such as makeup, lipstick and eyeliner. Furthermore, we explored the connections between tags, representing very popular content trends within the network. Finally, we uncover substantial differences in posting behavior of influencers and news pages when compared to regular users, observing they post more frequently and about more specific content, suggesting what may attract more engagement on Instagram.

Palavras-chave: Instagram, alt-text, social media, image classification, image tagging, complex networks, influencers

Referências

Rakesh Agrawal, Ramakrishnan Srikant, et al. 1994. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, Vol. 1215. Santiago, 487–499.

Wasim Ahmed, Josep Vidal-Alaball, Joseph Downing, and Francesc López Seguí. 2020. COVID-19 and the 5G Conspiracy Theory: Social Network Analysis of Twitter Data. Journal of Medical Internet Research 22, 5 (2020), e19458.

Camila S Araujo, Luiz P D Correa, Ana P C Silva, Raquel O Prates, and Wagner Meira. 2014. It is Not Just a Picture: Revealing Some User Practices in Instagram. In 9th Latin American Web Congress. 19–23.

Camila S Araújo, Wagner Meira, and Virgilio Almeida. 2016. Identifying Stereotypes in the Online Perception of Physical Attractivenes. In Proc. of the 8th Intl. Conf. of Social Informatics. Springer, 419–437.

Saeideh Bakhshi, David A. Shamma, and Eric Gilbert. 2014. Faces engage us: photos with faces attract more likes and comments on Instagram. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (Toronto, Canada) (CHI’14). ACM, NY, 965–974.

Vincent Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008. Fast Unfolding of Communities in Large Networks. Journal of Statistical Mechanics Theory and Experiment 2008 (04 2008).

Meeyoung Cha, Hamed Haddadi, Fabrício Benevenuto, and Krishna Gummadi. 2010. Measuring User Influence in Twitter: The Million Follower Fallacy. Proc. of the International AAAI Conf. on Web and Social Media 4, 1 (2010), 10–17.

Koyel Chakraborty, Siddhartha Bhattacharyya, and Rajib Bag. 2020. A Survey of Sentiment Analysis from Social Media Data. IEEE Transactions on Computational Social Systems 7, 2 (2020), 450–464.

P Chauhan, N Sharma, and G Sikka. 2020. The emergence of social media data and sentiment analysis in election prediction. Journal of Ambient Intelligence and Humanized Computing 12, 2 (2020), 2601.

Rion Brattig Correia, Lang Li, and Luis M Rocha. 2016. Monitoring potential drug interactions and reactions via network analysis of instagram user timelines. In Biocomputing 2016: Proc. of the Pacific Symposium. World Scientific, 492–503.

Emory J. Edwards, Michael Gilbert, Emily Blank, and Stacy M. Branham. 2023. How the Alt Text Gets Made: What Roles and Processes of Alt Text Creation Can Teach Us About Inclusive Imagery. ACM Transactions on Accessible Computing 16, 2, Article 18 (jul 2023), 28 pages.

Carlos H G Ferreira, Fabrício Murai, Ana P C , Jussara M Almeida, Martino Trevisan, Luca Vassio, Idilio Drago, and Marco Mellia. 2020. Unveiling Community Dynamics on Instagram Political Network. In Proc. of the 12th ACM Conf. on Web Science (NY). ACM, 231–240.

Yuheng Hu, Lydia Manikonda, and Subbarao Kambhampati. 2014. What We Instagram: A First Analysis of Instagram Photo Content and User Types. Proc. of the International AAAI Conf. on Web and Social Media 8, 1 (2014), 595–598.

C. Hutto and Eric Gilbert. 2014. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. Proc. of the International AAAI Conf. on Web and Social Media 8, 1 (2014), 216–225.

Simona Ibba, Matteo Orrù, Filippo Eros Pani, and Simone Porru. 2015. Hashtag of Instagram: From Folksonomy to Complex Network.. In KEOD. 279–284.

Jared Katzman, Angelina Wang, Morgan Scheuerman, Su L Blodgett, Kristen Laird, Hanna Wallach, and Solon Barocas. 2023. Taxonomizing and Measuring Representational Harms: A Look at Image Tagging. AAAI Conf. on Artificial Intelligence 37, 12 (2023), 14277–14285.

Mirella Lapata. 2006. Automatic evaluation of information ordering: Kendall’s tau. Computational Linguistics 32, 4 (2006), 471–484.

Sebastián Lozano and Ester Gutiérrez. 2018. A complex network analysis of global tourism flows. International Journal of Tourism Research 20, 5 (2018), 588–604.

Hafiz A M Malik. 2022. Complex Network Formation and Analysis of Online Social Media Systems. Computer Modeling in Engineering & Sciences 130, 3 (2022), 1737–1750.

Breno Matos, Fabrício Benevenuto, and Ana P C da Silva. 2021. A Nudez como Estereótipo de Beleza através das Lentes do Instagram. In X Brazilian Workshop on Social Network Analysis and Mining. 163–168.

Philipe Melo, Fabrício Benevenuto, Daniel Kansaon, Vitor Mafra, and Kaio Sá. 2021. Monitor de WhatsApp: Um sistema para checagem de fatos no combate à desinformação. In Anais do XXVII Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia’21). SBC, 79–82.

Caroline Muñoz and Terri Towner. 2017. The Image is the Message: Instagram Marketing and the 2016 Presidential Primary Season. Journal of Political Marketing 16 (06 2017).

László Nemes and Attila Kiss. 2021. Social media sentiment analysis based on COVID-19. Journal of Information and Telecommunication 5, 1 (2021), 1–15. Taylor & Francis.

Serge Nyawa, Dieudonné Tchuente, and Samuel Fosso-Wamba. 2022. COVID-19 vaccine hesitancy: a social media analysis using deep learning. Annals of Operations Research (2022).

Abubakr H. Ombabi, Wael Ouarda, and Adel M. Alimi. 2020. Deep learning CNN–LSTM framework for Arabic sentiment analysis using textual information shared in social networks. Social Network Analysis and Mining 10, 1 (2020), 53.

Khubaib A Qureshi, Rauf A S Malick, Muhammad Sabih, and Hocine Cherifi. 2021. Complex network and source inspired COVID-19 fake news classification on Twitter. IEEE Access 9 (2021), 139636–139656.

Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning. In Proc. of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1. ACL, 2556–2565.

Abigale Stangl, Meredith Ringel Morris, and Danna Gurari. 2020. “Person, Shoes, Tree. Is the Person Naked?” What People with Vision Impairments Want in Image Descriptions. In Proc. of the 2020 CHI Conf. on Human Factors in Computing Systems. ACM, 1–13.

Shaomei Wu, Jeffrey Wieland, Omid Farivar, and Julie Schiller. 2017. Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. In Proc. of the 2017 ACM Conf. on Computer Supported Cooperative Work and Social Computing (CSCW ’17). ACM, NY, 1180–1192.

Koosha Zarei, Reza Farahbakhsh, and Noël Crespi. 2019. Typification of Impersonated Accounts on Instagram. In IEEE 38th International Performance Computing and Communications Conf. (IPCCC). 1–6.