Language-Driven Graphs for Short Video Similarity - Yugoshi, Juliano Koji; Marcacini, Ricardo Marcondes (Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), 2025)
Comparing videos by similarity is a central task in modern video analysis, but its effectiveness depends critically on the chosen representation. While visual embeddings effectively capture appearance, they often lack semantic abstraction. Recently, Large Language Models (LLMs) have emerged as a promising way to generate rich textual descriptions, yet the structural properties they induce in video similarity spaces remain underexplored. We introduce a graph-based methodology to investigate these properties, systematically comparing the structure of similarity graphs derived from visual features, human-written text, and LLM-generated text. Our framework evaluates how well each graph preserves semantic consistency, both in immediate neighborhoods (local cohesion) and across longer paths (global organization). Our analysis reveals a fundamental trade-off: visual graphs exhibit high local purity but decay rapidly, whereas LLM-based graphs preserve superior global semantic coherence. We demonstrate that LLMs build an abstract space that prioritizes deep thematic links—such as grouping videos by concepts like ’stage performance’ across formal categories—over superficial purity. This structure offers a powerful, semantically organized alternative to the local cohesion of visual models.