Geração de audiodescrição de cena em transmissões DTV+
Resumo
Accessibility in audiovisual media remains an unequally exercised right: manual audio description is expensive, slow, and, in practice, offered for only a marginal fraction of television content. Objective: to present the Scene Describer (Descritor de Cenas), an automatic audio-description service built on top of the TV 3.0 (DTV+) platform. Method: the service captures the visual composition shown to the viewer, forwards the frames to a multimodal language model (Gemini 2.5 Flash) and returns a Brazilian-Portuguese description narrated by speech synthesis on the receiver itself; the architecture is fully decoupled via MQTT and integrates with the application orchestrator (App Operator) without rewriting the front-end, with a continuous narration mode and audio ducking to coexist with the program’s original audio. Results: on a running TV 3.0 stack, the measured median end-to-end latency is 4.8 s with a three-frame burst and 2.9 s with a single frame. The remote model call is bimodal: 20% of the calls exceed 10 s (P90 = 27.6 s), and this tail — not the median — is what rules out strictly live use. Across 56 descriptions, on-screen text was transcribed correctly in every case, with one movement hallucination and no invented entities. Conclusion: we discuss technical feasibility, perceived-latency limits, and the typical risks of generative models in the inclusive context, positioning the service as a baseline layer complementary to human audio description.
Palavras-chave:
Audio description, Accessibility, Digital TV, TV 3.0, DTV , Generative AI, Multimodal models, MQTT, Speech synthesis
Referências
Josh Achiam, Steven Adler, Sandhini Agarwal, et al. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023). arXiv:2303.08774 [cs.CL]
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a Visual Language Model for Few-Shot Learning. In Advances in Neural Information Processing Systems (NeurIPS).
Andrew Banks and Rahul Gupta. 2014. MQTT Version 3.1.1. Technical Report. OASIS Standard. OASIS Standard.
Virgínia P. Campos, Luiz M. G. Gonçalves, Wesnydy L. Ribeiro, Tiago M. U. Araújo, et al. 2023. Machine Generation of Audio Description for Blind and Visually Impaired People. ACM Transactions on Accessible Computing (TACCESS) 16, 2, Article 14 (2023). DOI: 10.1145/3590955
Lipisha Chaudhary, Trisha Mittal, Subhadra Gopalakrishnan, Ifeoma Nwogu, and Jaclyn Pytlarz. 2025. MCAD: Multimodal Context-Aware Audio Description Generation for Soccer. In 2025 IEEE International Symposium on Multimedia (ISM). IEEE.
Khang Dang, Grace Burke, Hamdi Korreshi, and Sooyeon Lee. 2026. Omnidirectional Audio Descriptions in Virtual Reality: A Conceptual Framework for Musical Performances Accessible to Blind and Low-Vision Users. ACM Transactions on Accessible Computing (TACCESS) 19, 2, Article 12 (2026). DOI: 10.1145/3813802
INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA. 2022. Censo 2022 | IBGE. [link]
Louise Fryer. 2016. An Introduction to Audio Description: A Practical Guide. Routledge.
Yingqiang Gao, Lukas Fischer, Alexa Lintner, and Sarah Ebling. 2025. Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies. In Findings of the Association for Computational Linguistics: NAACL 2025. Association for Computational Linguistics, Albuquerque, New Mexico, 471–490. DOI: 10.18653/v1/2025.findings-naacl.29
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2023. AutoAD II: The Sequel – Who, When, and What in Movie Audio Description. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE.
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2023. AutoAD: Movie Description in Context. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2024. AutoAD III: The Prequel – Back to the Pixels. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Lucy Jiang, Mahika Phutane, and Shiri Azenkot. 2023. Beyond Audio Description: Exploring 360∘ Video Accessibility with Blind and Low Vision Users Through Collaborative Creation. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23). ACM. DOI: 10.1145/3597638.3608381
Divy Kala, Eshika Khandelwal, and Makarand Tapaswi. 2025. What You See is What You Ask: Evaluating Audio Descriptions. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). arXiv:2510.00808 [cs.CV]
Rosiana Natalie, Ruei-Che Chang, Smitha Sheshadri, Anhong Guo, and Kotaro Hara. 2024. Audio Description Customization. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’24). ACM. DOI: 10.1145/3663548.3675617
Zheng Ning, Brianna L. Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, et al. 2024. SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM. DOI: 10.1145/3613904.3642632
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML).
Chris Richardson. 2018. Microservices Patterns: With Examples in Java. Manning Publications, Shelter Island, NY.
Xuyang Shen, Dong Li, Jinxing Zhou, Zhen Qin, Bowen He, Xiaodong Han, Aixuan Li, Yuchao Dai, Lingpeng Kong, Meng Wang, Yu Qiao, and Yiran Zhong. 2023. Fine-grained Audible Video Description. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv:2303.15616 [cs.CV]
Yuki Shimano, Yuya Kuwano, Masaki Takahashi, Hiroyuki Kaneko, et al. 2024. Semi-Automatic Production and Distribution System for Audio Descriptions in Live Baseball Broadcasts. In Proceedings of the 7th ACM International Workshop on Multimedia Content Analysis in Sports (MMSports ’24). ACM. DOI: 10.1145/3689061.3689069
Joel Snyder. 2014. The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description. American Council of the Blind.
Abigale Stangl, Shasta Ihorn, Yue-Ting Siu, Aditya Bodi, Mar Castanon, et al. 2023. The Potential of a Visual Dialogue Agent In a Tandem Automated Audio Description System for Videos. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23). ACM. DOI: 10.1145/3597638.3608402
Dirk Vander Wilt and Morwaread Mary Farbood. 2019. Automating Audio Description for Live Theater: Using Reference Recordings to Trigger Descriptive Tracks in Real Time. In Proceedings of the 14th International Audio Mostly Conference (AM ’19). ACM.
Hanlin Wang, Zhan Tong, Kecheng Zheng, Yujun Shen, and Limin Wang. 2025. Contextual AD Narration with Interleaved Multimodal Sequence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv:2403.12922 [cs.CV]
Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2024. AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description. In Computer Vision – ACCV 2024 (Lecture Notes in Computer Science, Vol. 15474). Springer, Singapore, 81–97. DOI: 10.1007/978-981-96-0908-6_5
Shuchang Xu, Xiaofu Jin, Gaurav Jain, Wenshuo Zhang, Huamin Qu, et al. 2026. Sonic Stage: Automatically Generating an Interactive Spatial Soundscape to Facilitate Dialogue Video Comprehension for Blind and Low Vision Viewers. In Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26). ACM. DOI: 10.1145/3772363.3798425
Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2024. MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a Visual Language Model for Few-Shot Learning. In Advances in Neural Information Processing Systems (NeurIPS).
Andrew Banks and Rahul Gupta. 2014. MQTT Version 3.1.1. Technical Report. OASIS Standard. OASIS Standard.
Virgínia P. Campos, Luiz M. G. Gonçalves, Wesnydy L. Ribeiro, Tiago M. U. Araújo, et al. 2023. Machine Generation of Audio Description for Blind and Visually Impaired People. ACM Transactions on Accessible Computing (TACCESS) 16, 2, Article 14 (2023). DOI: 10.1145/3590955
Lipisha Chaudhary, Trisha Mittal, Subhadra Gopalakrishnan, Ifeoma Nwogu, and Jaclyn Pytlarz. 2025. MCAD: Multimodal Context-Aware Audio Description Generation for Soccer. In 2025 IEEE International Symposium on Multimedia (ISM). IEEE.
Khang Dang, Grace Burke, Hamdi Korreshi, and Sooyeon Lee. 2026. Omnidirectional Audio Descriptions in Virtual Reality: A Conceptual Framework for Musical Performances Accessible to Blind and Low-Vision Users. ACM Transactions on Accessible Computing (TACCESS) 19, 2, Article 12 (2026). DOI: 10.1145/3813802
INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA. 2022. Censo 2022 | IBGE. [link]
Louise Fryer. 2016. An Introduction to Audio Description: A Practical Guide. Routledge.
Yingqiang Gao, Lukas Fischer, Alexa Lintner, and Sarah Ebling. 2025. Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies. In Findings of the Association for Computational Linguistics: NAACL 2025. Association for Computational Linguistics, Albuquerque, New Mexico, 471–490. DOI: 10.18653/v1/2025.findings-naacl.29
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2023. AutoAD II: The Sequel – Who, When, and What in Movie Audio Description. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE.
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2023. AutoAD: Movie Description in Context. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2024. AutoAD III: The Prequel – Back to the Pixels. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Lucy Jiang, Mahika Phutane, and Shiri Azenkot. 2023. Beyond Audio Description: Exploring 360∘ Video Accessibility with Blind and Low Vision Users Through Collaborative Creation. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23). ACM. DOI: 10.1145/3597638.3608381
Divy Kala, Eshika Khandelwal, and Makarand Tapaswi. 2025. What You See is What You Ask: Evaluating Audio Descriptions. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP). arXiv:2510.00808 [cs.CV]
Rosiana Natalie, Ruei-Che Chang, Smitha Sheshadri, Anhong Guo, and Kotaro Hara. 2024. Audio Description Customization. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’24). ACM. DOI: 10.1145/3663548.3675617
Zheng Ning, Brianna L. Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, et al. 2024. SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM. DOI: 10.1145/3613904.3642632
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML).
Chris Richardson. 2018. Microservices Patterns: With Examples in Java. Manning Publications, Shelter Island, NY.
Xuyang Shen, Dong Li, Jinxing Zhou, Zhen Qin, Bowen He, Xiaodong Han, Aixuan Li, Yuchao Dai, Lingpeng Kong, Meng Wang, Yu Qiao, and Yiran Zhong. 2023. Fine-grained Audible Video Description. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv:2303.15616 [cs.CV]
Yuki Shimano, Yuya Kuwano, Masaki Takahashi, Hiroyuki Kaneko, et al. 2024. Semi-Automatic Production and Distribution System for Audio Descriptions in Live Baseball Broadcasts. In Proceedings of the 7th ACM International Workshop on Multimedia Content Analysis in Sports (MMSports ’24). ACM. DOI: 10.1145/3689061.3689069
Joel Snyder. 2014. The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description. American Council of the Blind.
Abigale Stangl, Shasta Ihorn, Yue-Ting Siu, Aditya Bodi, Mar Castanon, et al. 2023. The Potential of a Visual Dialogue Agent In a Tandem Automated Audio Description System for Videos. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23). ACM. DOI: 10.1145/3597638.3608402
Dirk Vander Wilt and Morwaread Mary Farbood. 2019. Automating Audio Description for Live Theater: Using Reference Recordings to Trigger Descriptive Tracks in Real Time. In Proceedings of the 14th International Audio Mostly Conference (AM ’19). ACM.
Hanlin Wang, Zhan Tong, Kecheng Zheng, Yujun Shen, and Limin Wang. 2025. Contextual AD Narration with Interleaved Multimodal Sequence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv:2403.12922 [cs.CV]
Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, and Andrew Zisserman. 2024. AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description. In Computer Vision – ACCV 2024 (Lecture Notes in Computer Science, Vol. 15474). Springer, Singapore, 81–97. DOI: 10.1007/978-981-96-0908-6_5
Shuchang Xu, Xiaofu Jin, Gaurav Jain, Wenshuo Zhang, Huamin Qu, et al. 2026. Sonic Stage: Automatically Generating an Interactive Spatial Soundscape to Facilitate Dialogue Video Comprehension for Blind and Low Vision Viewers. In Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26). ACM. DOI: 10.1145/3772363.3798425
Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2024. MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
Publicado
27/10/2026
Como Citar
ARAÚJO, Lucas Álefe Estevo de; FARIAS, Flávio Miranda de; MUCHALUAT-SAADE, Débora C..
Geração de audiodescrição de cena em transmissões DTV+. In: WORKSHOP FUTURO DA TV DIGITAL INTERATIVA - SIMPÓSIO BRASILEIRO DE SISTEMAS MULTIMÍDIA E WEB (WEBMEDIA), 32. , 2026, Lavras/MG.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 255-260.
ISSN 2596-1683.
DOI: https://doi.org/10.5753/webmedia_estendido.2026.32518.
