Automatic 3D animation generation for Sign Language: A case study of Sign Language Machine Translation - Martins, Luisa; Ferreira, Stênio; Maritan, Tiago (Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia), 2025)
Digital accessibility remains a critical challenge for the Deaf community, especially in Brazil, when it comes to universalizing Brazilian Sign Language (Libras) content through different media. Traditionally, animation pipelines for Sign Language videos, including those used in the VLibras suite, are frequently slow, costly, and limited in expressiveness. This work presents a novel approach that automates the transformation of human-generated Libras videos into expressive 3D animations using deep learning-based motion transfer techniques to be applied in VLibras workflow and extend its capability in order to improve the demanded time for implementation and cost dramatically. The system leverages recent progresses in motion capture, pose estimation, and animation retargeting to enable efficient and real-time animation of human based 3D models. The present study discusses the system architecture, automation pipeline, and the potential of value creation for the Deaf community through a survey and evaluation process. Results indicate that the approach used supported the improvement on realism, enhancement on a more immersive experience, acceleration of the development and potentially high quality compared to traditional methods. Future developments on fine-tuning the model and the generation of anthropomorphic enhanced 3D models with a better collision detection capability can enable a scalable path toward a more inclusive media content for the Deaf population in Brazil.