Médicos, Máquinas e Marketing Multimodal: Avaliando VLMs para Anotação Especializada de Anúncios Relacionados à Saúde
Resumo
Identifying medicalization in advertisements is a task that requires specialized knowledge and the joint interpretation of visual and textual elements. Although Vision-Language Models (VLMs) have shown significant advances in various multimodal tasks, there is still limited evidence regarding their ability to replicate specialized judgments in highly complex domains. In this work, we investigate the feasibility of using VLMs to annotate medicalization in advertisements from the Meta Ad Library. From a dataset of 7,167 ads containing text and images, we selected 300 examples that require specialized knowledge to assess their level of medicalization. The advertisements were annotated by three human experts and by three VLMs from different families, allowing for the comparison of human agreement, inter-model agreement, and the alignment between these groups. The results show that VLMs achieve agreement levels close to those observed among human experts in identifying medicalized content, suggesting that they are already capable of capturing a substantial portion of the criteria used by experts in this task. However, important differences are observed regarding the intensity attributed to the most ambiguous cases. Additionally, we identified substantial variability among models in aspects such as consistency across repeated runs and adherence to the annotation instructions. Finally, we conducted a qualitative analysis of the cases with the highest agreement and disagreement, characterizing the types of ads that facilitate and hinder automatic annotation. The results suggest that VLMs can serve as support for building specialized datasets for monitoring medicalization elements in advertisements, while also highlighting important limitations that must be considered in such applications.
Palavras-chave:
Inteligência Artificial, Visual-Language-models, VLM, anúncios, Ad Library, dados de saúde, medicalização, Meta
Referências
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35 (2022), 23716–23736.
ANVISA. 2008. RESOLUÇÃO-RDC Nº 96, DE 17 DE DEZEMBRO DE 2008. Dispõe sobre a propaganda, publicidade, informação e outras práticas cujo objetivo seja a divulgação ou promoção comercial de medicamentos.
Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. 2025. Health-Bench: Evaluating Large Language Models Towards Improved Human Health. arXiv:2505.08775 [cs.CL] [link]
Merve Astekin, Max Hort, and Leon Moonen. 2024. An exploratory study on how non-determinism in large language models affects log parsing. In Proceedings of the ACM/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering. 13–18.
Merve Astekin, Max Hort, and Leon Moonen. 2024. An Exploratory Study on How Non-Determinism in Large Language Models Affects Log Parsing. In Proceedings of the ACM/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering (Lisbon, Portugal) (InteNSE ’24). Association for Computing Machinery, New York, NY, USA, 13–18. DOI: 10.1145/3643661.3643952
Peter Conrad. 2007. The medicalization of society: On the transformation of human conditions into treatable disorders. Vol. 14. Johns Hopkins University Press Baltimore.
Conselho Federal de Medicina. 2023. Resolução CFM nº 2.336/2023. Dispõe sobre publicidade e propaganda médicas. Diário Oficial da União, Brasília, DF, nº 175, 13 set. 2023. Disponível em: [link].
Washington Cunha, Celso França, Guilherme Fonseca, Leonardo Rocha, and Marcos André Gonçalves. 2023. An Effective, Efficient, and Scalable Confidence-Based Instance Selection Framework for Transformer-Based Text Classification. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 665–674. DOI: 10.1145/3539618.3591638
Washington Cunha, Felipe Viegas, Celso França, Guilherme Fonseca, Leonardo Rocha, and Marcos André Gonçalves. 2025. A Noise-Oriented and Redundancy-Aware Instance Selection Framework. ACM Transactions on Information Systems 43, 1 (2025). DOI: 10.1145/3701769
Jonatas Henrique dos Santos, Julio CS Reis, Philipe de Freitas Melo, João Francisco Hecksher Olivetti, Thales Henrique Silva, Matheus Gontijo Guimaraes, Glaucio de Souza, Marcos Goncalves, Fabricio Benevenuto, Filipe Belchior Bessa Zanovello, et al. 2026. Whatsapp vaccine discourse (whavax): An expert-annotated dataset and benchmark for health misinformation detection. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 20. 2768–2779.
Laura Edelson, Tobias Lauinger, and Damon McCoy. 2020. A Security Analysis of the Facebook Ad Library. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 661–678. DOI: 10.1109/SP40000.2020.00084
David Fernandes, Edleno Silva de Moura, Berthier A. Ribeiro-Neto, Altigran Soares da Silva, and Marcos André Gonçalves. 2007. Computing Block Importance for Searching on Web Sites. In Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management. ACM, 165–174. DOI: 10.1145/1321440.1321466
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences 120, 30 (2023), e2305016120.
Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022).
Rebecca Soskin Hicks, Mikhail Trofimov, Dominick Lim, Rahul K Arora, Foivos Tsimpourlas, Preston Bowman, Michael Sharman, Chi Tong, Kavin Karthik, Arnav Dugar, et al. 2026. HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats. arXiv preprint arXiv:2604.27470 (2026).
Victor Le Pochat, Laura Edelson, Tom Van Goethem, Wouter Joosen, Damon McCoy, and Tobias Lauinger. 2022. An Audit of Facebook’s Political Ad Policy Enforcement. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, 607–624.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890 (2023).
Bryan A Liang and Tim Mackey. 2011. Direct-to-consumer advertising with interactive internet media: global regulation and public health issues. JAMA 305, 8 (2011), 824–825.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems 36 (2023), 34892–34916.
Jialin Liu and Siru Liu. 2025. HealthBench: Advancing AI evaluation in healthcare, but not yet clinically ready. DIGITAL HEALTH 11 (2025), 20552076251390447. DOI: 10.1177/20552076251390447
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med-Flamingo: a Multimodal Medical Few-shot Learner. In Proceedings of the 3rd Machine Learning for Health Symposium (Proceedings of Machine Learning Research, Vol. 225), Stefan Hegselmann, Antonio Parziale, Divya Shanmugam, Shengpu Tang, Mercy Nyamewaa Asiedu, Serina Chang, Tom Hartvigsen, and Harvineet Singh (Eds.). PMLR, 353–367.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, and Jure Leskovec. 2023. Med-Flamingo: A Multimodal Medical Few-shot Learner. (July 2023). [link] arXiv:2307.15189.
Niklas Muennighoff, Alexander Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. 2023. Scaling Data-Constrained Language Models. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 50358–50376. [link]
Amanda Myntti, Erik Henriksson, Veronika Laippala, and Sampo Pyysalo. 2025. Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation. In Second Conference on Language Modeling. [link]
Rohit Patel, Alexandre Rezende, and Steven McClain. 2026. GIM: Evaluating models via tasks that integrate multiple cognitive domains. arXiv:2605.18663 [cs.AI] [link]
Cheng Peng, Kai Zhang, Mengxian Lyu, Hongfang Liu, Lichao Sun, and Yonghui Wu. 2025. Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning. arXiv preprint arXiv:2505.17436 (2025).
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748–8763. [link]
Pedro Miguel Lima de Sousa Reis. 2022. Data Labeling tools for Computer Vision: a Review.
Aryan Sajith and Krishna Chaitanya Rao Kathala. 2025. Is Training Data Quality or Quantity More Impactful to Small Language Model Performance. In Neural Computing for Advanced Applications, Haijun Zhang, Kim Fung Tsang, Fu Lee Wang, Tianyong Hao, Zenghui Wang, Zhou Wu, Zhao Zhang, and Kevin Hung (Eds.). Springer Nature Singapore, Singapore, 75–87.
Hajar Sakai and Sarah S Lam. 2026. Large Language Models for Health Care Text Classification: Systematic Review. JMIR AI 5 (11 Feb 2026), e79202. DOI: 10.2196/79202
Silvana Secinaro, Davide Calandra, A Secinaro, V Muthurangu, and Paolo Biancone. 2021. The role of artificial intelligence in healthcare: a structured literature review. BMC Medical Informatics and Decision Making 21, 1 (2021), 125. DOI: 10.1186/s12911-021-01488-9
Cora Silberschneider, Samuel S. Guimarães, Fabrício Benevenuto, and Márcio Silva. 2022. Characterizing Brazilian Political Ads on Facebook. In Proceedings of the Brazilian Symposium on Multimedia and the Web (Curitiba, Brazil) (WebMedia ’22). Association for Computing Machinery, New York, NY, USA, 47–54. DOI: 10.1145/3539637.3557935
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H Chen, Nigam H Shah, Sami Lachgar, Philip Andrew Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Agüera Y Arcas, Nenad Tomašev, Yun Liu, Renee Wong, Christopher Semturs, S Sara Mahdavi, Joelle K Barral, Dale R Webster, Greg S Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan. 2025. Toward expert-level medical question answering with large language models. Nat. Med. 31, 3 (March 2025), 943–950.
Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. 2025. The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 4195–4206. DOI: 10.18653/v1/2025.naacl-long.211
Haozhan Sun, Chenchen Xu, and Hanna Suominen. 2021. Analyzing the Granularity and Cost of Annotation in Clinical Sequence Labeling. arXiv preprint arXiv:2108.09913 (2021). [link]
Petter Törnberg. 2024. Best practices for text annotation with large language models. arXiv preprint arXiv:2402.05129 (2024).
Felipe Viegas, Sérgio Canuto, Christian Gomes, Washington Cunha, Thierson Rosa, Sabir Ribas, Leonardo Rocha, and Marcos André Gonçalves. 2019. CluWords: Exploiting Semantic Word Clustering Representation for Enhanced Topic Modeling. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. ACM, 753–761. DOI: 10.1145/3289600.3291032
Felipe Viegas, Washington Cunha, Christian Gomes, Antônio Pereira, Leonardo Rocha, and Marcos Goncalves. 2020. CluHTM: Semantic Hierarchical Topic Modeling Based on CluWords. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 8138–8150. DOI: 10.18653/v1/2020.acl-main.724
Felipe Viegas, Washington Cunha, Christian Gomes, Leonardo Rocha, and Marcos Goncalves. 2025. Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics. Computational Linguistics (2025).
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. 2025. MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 15134–15186. DOI: 10.18653/v1/2025.acl-long.736
Marco Zenone, Nora Kenworthy, and Nason Maani. 2023. The social media industry as a commercial determinant of health. International Journal of Health Policy and Management 12, 1 (2023), 1–4.
Marco Zenone, Jeremy Snyder, Jean-Christophe Bélisle-Pipon, Timothy Caulfield, May CI van Schalkwyk, and Nason Maani. 2023. Advertising alternative cancer treatments and approaches on Meta social media platforms: content analysis. JMIR Infodemiology 3, 1 (2023), e43548.
Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D Davison, Hui Ren, et al. 2024. A generalist vision–language foundation model for diverse biomedical tasks. Nature Medicine (2024), 1–13.
Zhihao Zhang, Yiran Zhang, Xiyue Zhou, Liting Huang, Imran Razzak, Preslav Nakov, and Usman Naseem. 2025. From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 24245–24260. DOI: 10.18653/v1/2025.findings-emnlp.1316
Jie Zheng et al. 2018. Clinical text annotation – what factors are associated with the cost of time?. In AMIA Annual Symposium Proceedings, Vol. 2018. American Medical Informatics Association, 1552–1560.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems 36 (2023), 46595–46623.
ANVISA. 2008. RESOLUÇÃO-RDC Nº 96, DE 17 DE DEZEMBRO DE 2008. Dispõe sobre a propaganda, publicidade, informação e outras práticas cujo objetivo seja a divulgação ou promoção comercial de medicamentos.
Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. 2025. Health-Bench: Evaluating Large Language Models Towards Improved Human Health. arXiv:2505.08775 [cs.CL] [link]
Merve Astekin, Max Hort, and Leon Moonen. 2024. An exploratory study on how non-determinism in large language models affects log parsing. In Proceedings of the ACM/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering. 13–18.
Merve Astekin, Max Hort, and Leon Moonen. 2024. An Exploratory Study on How Non-Determinism in Large Language Models Affects Log Parsing. In Proceedings of the ACM/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering (Lisbon, Portugal) (InteNSE ’24). Association for Computing Machinery, New York, NY, USA, 13–18. DOI: 10.1145/3643661.3643952
Peter Conrad. 2007. The medicalization of society: On the transformation of human conditions into treatable disorders. Vol. 14. Johns Hopkins University Press Baltimore.
Conselho Federal de Medicina. 2023. Resolução CFM nº 2.336/2023. Dispõe sobre publicidade e propaganda médicas. Diário Oficial da União, Brasília, DF, nº 175, 13 set. 2023. Disponível em: [link].
Washington Cunha, Celso França, Guilherme Fonseca, Leonardo Rocha, and Marcos André Gonçalves. 2023. An Effective, Efficient, and Scalable Confidence-Based Instance Selection Framework for Transformer-Based Text Classification. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 665–674. DOI: 10.1145/3539618.3591638
Washington Cunha, Felipe Viegas, Celso França, Guilherme Fonseca, Leonardo Rocha, and Marcos André Gonçalves. 2025. A Noise-Oriented and Redundancy-Aware Instance Selection Framework. ACM Transactions on Information Systems 43, 1 (2025). DOI: 10.1145/3701769
Jonatas Henrique dos Santos, Julio CS Reis, Philipe de Freitas Melo, João Francisco Hecksher Olivetti, Thales Henrique Silva, Matheus Gontijo Guimaraes, Glaucio de Souza, Marcos Goncalves, Fabricio Benevenuto, Filipe Belchior Bessa Zanovello, et al. 2026. Whatsapp vaccine discourse (whavax): An expert-annotated dataset and benchmark for health misinformation detection. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 20. 2768–2779.
Laura Edelson, Tobias Lauinger, and Damon McCoy. 2020. A Security Analysis of the Facebook Ad Library. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 661–678. DOI: 10.1109/SP40000.2020.00084
David Fernandes, Edleno Silva de Moura, Berthier A. Ribeiro-Neto, Altigran Soares da Silva, and Marcos André Gonçalves. 2007. Computing Block Importance for Searching on Web Sites. In Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management. ACM, 165–174. DOI: 10.1145/1321440.1321466
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences 120, 30 (2023), e2305016120.
Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022).
Rebecca Soskin Hicks, Mikhail Trofimov, Dominick Lim, Rahul K Arora, Foivos Tsimpourlas, Preston Bowman, Michael Sharman, Chi Tong, Kavin Karthik, Arnav Dugar, et al. 2026. HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats. arXiv preprint arXiv:2604.27470 (2026).
Victor Le Pochat, Laura Edelson, Tom Van Goethem, Wouter Joosen, Damon McCoy, and Tobias Lauinger. 2022. An Audit of Facebook’s Political Ad Policy Enforcement. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, 607–624.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890 (2023).
Bryan A Liang and Tim Mackey. 2011. Direct-to-consumer advertising with interactive internet media: global regulation and public health issues. JAMA 305, 8 (2011), 824–825.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems 36 (2023), 34892–34916.
Jialin Liu and Siru Liu. 2025. HealthBench: Advancing AI evaluation in healthcare, but not yet clinically ready. DIGITAL HEALTH 11 (2025), 20552076251390447. DOI: 10.1177/20552076251390447
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med-Flamingo: a Multimodal Medical Few-shot Learner. In Proceedings of the 3rd Machine Learning for Health Symposium (Proceedings of Machine Learning Research, Vol. 225), Stefan Hegselmann, Antonio Parziale, Divya Shanmugam, Shengpu Tang, Mercy Nyamewaa Asiedu, Serina Chang, Tom Hartvigsen, and Harvineet Singh (Eds.). PMLR, 353–367.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, and Jure Leskovec. 2023. Med-Flamingo: A Multimodal Medical Few-shot Learner. (July 2023). [link] arXiv:2307.15189.
Niklas Muennighoff, Alexander Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. 2023. Scaling Data-Constrained Language Models. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 50358–50376. [link]
Amanda Myntti, Erik Henriksson, Veronika Laippala, and Sampo Pyysalo. 2025. Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation. In Second Conference on Language Modeling. [link]
Rohit Patel, Alexandre Rezende, and Steven McClain. 2026. GIM: Evaluating models via tasks that integrate multiple cognitive domains. arXiv:2605.18663 [cs.AI] [link]
Cheng Peng, Kai Zhang, Mengxian Lyu, Hongfang Liu, Lichao Sun, and Yonghui Wu. 2025. Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning. arXiv preprint arXiv:2505.17436 (2025).
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748–8763. [link]
Pedro Miguel Lima de Sousa Reis. 2022. Data Labeling tools for Computer Vision: a Review.
Aryan Sajith and Krishna Chaitanya Rao Kathala. 2025. Is Training Data Quality or Quantity More Impactful to Small Language Model Performance. In Neural Computing for Advanced Applications, Haijun Zhang, Kim Fung Tsang, Fu Lee Wang, Tianyong Hao, Zenghui Wang, Zhou Wu, Zhao Zhang, and Kevin Hung (Eds.). Springer Nature Singapore, Singapore, 75–87.
Hajar Sakai and Sarah S Lam. 2026. Large Language Models for Health Care Text Classification: Systematic Review. JMIR AI 5 (11 Feb 2026), e79202. DOI: 10.2196/79202
Silvana Secinaro, Davide Calandra, A Secinaro, V Muthurangu, and Paolo Biancone. 2021. The role of artificial intelligence in healthcare: a structured literature review. BMC Medical Informatics and Decision Making 21, 1 (2021), 125. DOI: 10.1186/s12911-021-01488-9
Cora Silberschneider, Samuel S. Guimarães, Fabrício Benevenuto, and Márcio Silva. 2022. Characterizing Brazilian Political Ads on Facebook. In Proceedings of the Brazilian Symposium on Multimedia and the Web (Curitiba, Brazil) (WebMedia ’22). Association for Computing Machinery, New York, NY, USA, 47–54. DOI: 10.1145/3539637.3557935
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H Chen, Nigam H Shah, Sami Lachgar, Philip Andrew Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Agüera Y Arcas, Nenad Tomašev, Yun Liu, Renee Wong, Christopher Semturs, S Sara Mahdavi, Joelle K Barral, Dale R Webster, Greg S Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan. 2025. Toward expert-level medical question answering with large language models. Nat. Med. 31, 3 (March 2025), 943–950.
Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. 2025. The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 4195–4206. DOI: 10.18653/v1/2025.naacl-long.211
Haozhan Sun, Chenchen Xu, and Hanna Suominen. 2021. Analyzing the Granularity and Cost of Annotation in Clinical Sequence Labeling. arXiv preprint arXiv:2108.09913 (2021). [link]
Petter Törnberg. 2024. Best practices for text annotation with large language models. arXiv preprint arXiv:2402.05129 (2024).
Felipe Viegas, Sérgio Canuto, Christian Gomes, Washington Cunha, Thierson Rosa, Sabir Ribas, Leonardo Rocha, and Marcos André Gonçalves. 2019. CluWords: Exploiting Semantic Word Clustering Representation for Enhanced Topic Modeling. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. ACM, 753–761. DOI: 10.1145/3289600.3291032
Felipe Viegas, Washington Cunha, Christian Gomes, Antônio Pereira, Leonardo Rocha, and Marcos Goncalves. 2020. CluHTM: Semantic Hierarchical Topic Modeling Based on CluWords. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 8138–8150. DOI: 10.18653/v1/2020.acl-main.724
Felipe Viegas, Washington Cunha, Christian Gomes, Leonardo Rocha, and Marcos Goncalves. 2025. Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics. Computational Linguistics (2025).
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. 2025. MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 15134–15186. DOI: 10.18653/v1/2025.acl-long.736
Marco Zenone, Nora Kenworthy, and Nason Maani. 2023. The social media industry as a commercial determinant of health. International Journal of Health Policy and Management 12, 1 (2023), 1–4.
Marco Zenone, Jeremy Snyder, Jean-Christophe Bélisle-Pipon, Timothy Caulfield, May CI van Schalkwyk, and Nason Maani. 2023. Advertising alternative cancer treatments and approaches on Meta social media platforms: content analysis. JMIR Infodemiology 3, 1 (2023), e43548.
Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D Davison, Hui Ren, et al. 2024. A generalist vision–language foundation model for diverse biomedical tasks. Nature Medicine (2024), 1–13.
Zhihao Zhang, Yiran Zhang, Xiyue Zhou, Liting Huang, Imran Razzak, Preslav Nakov, and Usman Naseem. 2025. From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 24245–24260. DOI: 10.18653/v1/2025.findings-emnlp.1316
Jie Zheng et al. 2018. Clinical text annotation – what factors are associated with the cost of time?. In AMIA Annual Symposium Proceedings, Vol. 2018. American Medical Informatics Association, 1552–1560.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems 36 (2023), 46595–46623.
Publicado
27/10/2026
Como Citar
OLIVETTI, João F. H. et al.
Médicos, Máquinas e Marketing Multimodal: Avaliando VLMs para Anotação Especializada de Anúncios Relacionados à Saúde. In: BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 32. , 2026, Lavras/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 309-318.
DOI: https://doi.org/10.5753/webmedia.2026.30857.
