Marcelo Garcia Manzato
Rudinei Goularte
ABSTRACT
User tags are an important source of information that can be used to gather semantic data about the content, reducing the semantic gap and the restrictive domain of automatic indexing approaches. In this paper, we propose an automatic technique for semantic annotation of multimedia content based on collaborative user tags. Our technique faces some of the challenges of using user-generated terms, such as noise and incompleteness. Based on the actual context of a multimedia item and the co-occurrence of concepts and tags from the training set, we are able to predict semantic concepts for new items without the need of complex multimedia indexing techniques. We describe the results of our approach with an evaluation of our algorithm using a large scale dataset composed of images and user tags.
- R. A. Batal and P. Mulhem. MRIM-LIG at ImageCLEF 2010 Visual Concept Detection and Annotation task. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- D. Brezeale and D. J. Cook. Automatic Video Classification: A Survey of the Literature. IEEE Transactions on Systems, Man, and Cybernetics, 38(3):416--430, 2007. Google ScholarDigital Library
- I. Cantador, M. Szomszor, H. Alani, M. Fernández, and P. Castells. Enriching Ontological User Profiles with Tagging History for Multi-Domain Recommendations. In 1st International Workshop on Collective Semantics: Collective Intelligence & the Semantic Web (CISWeb 2008), Tenerife, Spain, 2008.Google Scholar
- C. Cattuto, D. Benz, A. Hotho, and G. Stumme. Semantic Grounding of Tag Relatedness in Social Bookmarking Systems. In A. P. Sheth, S. Staab, M. Dean, M. Paolucci, D. Maynard, T. W. Finin, and K. Thirunarayan, editors, The Semantic Web -- ISWC 2008, volume 5318 of Lecture Notes in Computer Science, pages 615--631, Berlin/Heidelberg, 2008. Springer. Google ScholarDigital Library
- B. Daroczy, I. Petras, A. A. Benczur, D. Nemeskey, and R. Pethes. SZTAKI @ ImageCLEF 2010. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- I. Dimitrovski, D. Kocev, S. Loskovska, and S. Dzeroski. Detection of Visual COncepts and Annotation of Images using Predictive Clustering Trees. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- A. Fakeri-Tabrizi, S. Tollari, N. Usunier, M. R. Amini, and P. Gallinari. UPMC/LIP6 at ImageCLEF annotation 2010. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- J. S. Hare, P. H. Lewis, P. G. B. Enser, and C. J. S. A linear-algebraic technique with an application in semantic image retrieval. In Proceedings of the International Conference on Image and Video Retrieval, pages 31--40. Springer, 2006. Google ScholarDigital Library
- M. J. Huiskes and M. S. Lew. The mir flickr retrieval evaluation. In MIR '08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- Y. A. Lacerda, H. F. de Figueiredo, C. de Souza Baptista, and A. C. de Paiva. Expandindo e utilizando informacoes de contexto para a sugestao de anotacoes de fotografias digitais. In Proceedings of the 14th Brazilian Symposium on Multimedia and the Web, WebMedia '08, pages 162--169, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- W. Li, J. Min, and G. J. F. Jones. A Text-Based Approach to the ImageCLEF 2010 Photo Annotation Task. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- C. D. Manning, P. Raghavan, and H. Schtze. Introduction to Information Retrieval. Cambridge University Press, New York, NY, USA, 2008. Google ScholarDigital Library
- E. Mbanya, C. Hentschel, S. Gerke, M. Liu, A. Nurnberger, and P. Ndjiki-Nya. Augmenting Bag-of-Words - Category Specific Features and Concept Reasoning. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- T. Mensink, G. Csurka, F. Perronnin, J. Sánchez, and J. Verbeek. LEAR and XRCE's participation to Visual Concept Detection Task -- ImageCLEF 2010. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- N. Motohashi, R. Izawa, and T. Takagi. Meiji University at ImageCLEF2010 Visual Concept Detection and Annotation Task. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- M. Naaman, S. Harada, Q. Wang, H. Garcia-Molina, and A. Paepcke. Context data in geo-referenced digital photo collections. In Proceedings of the 12th annual ACM international conference on Multimedia, MULTIMEDIA '04, pages 196--203, New York, NY, USA, 2004. ACM. Google ScholarDigital Library
- S. Paris and H. Glotin. Linear SVM for LSIS Pyramidal Multi-Level Visual only Concept Detection in CLEF 2010 Challenge. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- S. N. Patel and G. D. Abowd. The ContextCam: Automated Point of Capture Video Annotation. In F. Khendek and R. Dssouli, editors, UbiComp 2004: Ubiquitous Computing, volume 3205 of Lecture Notes in Computer Science, pages 301--318. Springer, 2004.Google Scholar
- C. Rasche and C. Vertan. A Novel Structural-Description Approach for Image Retrieval. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- C. J. V. Rijsbergen. Information Retrieval. Butterworth-Heinemann, 2nd edition, 1979. Google ScholarDigital Library
- H. Sahbi and X. Li. TELECOM ParisTech at ImageCLEF 2010 Photo Annotation Task: COmbining Tags and Visual Features for Learning-Based Image Annotation. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- F. Sebastiani. Machine learning in automated text categorization. ACM Comput. Surv., 34, 2002. Google ScholarDigital Library
- R. Shi, C.-H. Lee, and T.-S. Chua. Enhancing image annotation by integrating concept ontology and text-based bayesian learning model. In Proceedings of the 15th international conference on Multimedia, MULTIMEDIA '07, pages 341--344, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-Based Image Retrieval at the End of the Early Years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349--1380, 2000. Google ScholarDigital Library
- M. Srikanth, J. Varner, M. Bowden, and D. Moldovan. Exploiting ontologies for automatic image annotation. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '05, pages 552--558, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- M. Stanek and O. Maier. The Wroclaw University of Technology Participation at ImageCLEF 2010 Photo Annotation Track. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- P. R. Sun. Using information content to evaluate semantic similarity in a taxonomy. In In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448--453, 1995. Google ScholarDigital Library
- J. Tang, R. Hong, S. Yan, T.-S. Chua, G.-J. Qi, and R. Jain. Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images. ACM Transactions on Intelligent Systems and Technology, 2:14:1--14:15, 2011. Google ScholarDigital Library
- J. Tang, S. Yan, R. Hong, G.-J. Qi, and T.-S. Chua. Inferring semantic concepts from community-contributed images and noisy tags. In W. Gao, Y. Rui, A. Hanjalic, C. Xu, E. G. Steinbach, A. El-Saddik, and M. X. Zhou, editors, ACM Multimedia, pages 223--232. ACM, 2009. Google ScholarDigital Library
- K. E. A. van de Sande and T. Gevers. The University of Amsterdam's Concept Detection System at ImageClEF 2010. In Working Notes of CLEF 2010, Padua, Italy, 2010. Google ScholarDigital Library
- J. Verbeek, M. Guillaumin, T. Mensink, and C. Schmid. Image Annotation with TagProp on the MIRFLICKR set. In 11th ACM International Conference on Multimedia Information Retrieval (MIR '10), pages 537--546, Philadephia, USA, 2010. ACM Press. Google ScholarDigital Library
- W. Viana, A. Miron, B. Moisuc, J. Gensel, M. Villanova-Oliver, and H. Martin. Towards the semantic and context-aware management of mobile multimedia. Multimedia Tools and Applications, 53:391--429, 2011. Google ScholarDigital Library
- Z. Wu and M. Palmer. Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Association for Computational Linguistics, pages 133--138, Stroudsburg, PA, USA, 1994. Association for Computational Linguistics. Google ScholarDigital Library
- D. Zhang, B. Liu, C. Sun, and X. Wang. Random Sampling Image to Class DIstance for Photo Annotation. In Working Notes of CLEF 2010, Padua, Italy, 2010.Google Scholar
- S. Zhu, G. Wang, C.-W. Ngo, and Y.-G. Jiang. On the sampling of web images for learning visual concept classifiers. In Proceedings of the ACM International Conference on Image and Video Retrieval, CIVR '10, pages 50--57, New York, USA, 2010. Google ScholarDigital Library
Index Terms
- Automatic annotation of tagged content using predefined semantic concepts
Recommendations
Semantic annotation of personal video content using an image folksonomy
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingThe increasing popularity of user-generated content (UGC) requires effective annotation techniques in order to facilitate precise content search and retrieval. In this paper, we propose a new approach for the semantic annotation of personal video ...
Automatic tag expansion using visual similarity for photo sharing websites
In this paper we present an automatic photo tag expansion method designed for photo sharing websites. The purpose of the method is to suggest tags that are relevant to the visual content of a given photo at upload time. Both textual and visual cues are ...
Semantic concept detection for user-generated video content using a refined image folksonomy
MMM'10: Proceedings of the 16th international conference on Advances in Multimedia ModelingThe automatic detection of semantic concepts is a key technology for enabling efficient and effective video content management. Conventional techniques for semantic concept detection in video content still suffer from several interrelated issues: the ...
Comments