LLM-Generated Dataset for Speech-Driven 3D Facial Animation Models with Text-Controlled Expressivity - Corrêa, Pedro Rodrigues; Costa, Paula Dornhofer Paro (Anais Estendidos da Conference on Graphics, Patterns and Images (SIBGRAPI); 2025: Anais Estendidos da XXXVIII Conference on Graphics, Patterns and ImagesDO - 10.5753/sibgrapi.est.2025.38323, 2025)
The field of speech-driven 3D facial animation has a fundamental limitation regarding style control by natural language: scarcity of datasets that pair nuanced language with direct, controllable facial parameters. To address this bottleneck, we introduce two core contributions: (1) a large-scale, synthetically generated dataset of over 53,000 items that link emotional transcripts, descriptive tags, and explicit facial descriptions to 51-dimensional blendshape vectors; and (2) a CLIP-based alignment module that learns a shared semantic manifold between this text and the corresponding expressions. Our model employs a Transformer-based blendshape autoencoder and a frozen CLIP text encoder, optimized via a cross-modal objective. We evaluate the structure of this learned manifold using t-SNE projections and latent space traversals, which demonstrate semantically coherent clustering and smooth, plausible transitions in expression. We release our dataset specification, generation methodology, and training recipe to facilitate reproducible research in text-driven facial animation.