Uma Ferramenta de Autoria para Interação Multimodal em Aplicações NCL 4.0

• Pedro Alves Valentim UFF
• Débora C. Muchaluat-Saade UFF

DOI: https://doi.org/10.5753/webmedia_estendido.2024.244590

Resumo

Digital television has become an increasingly important platform for communication and entertainment, especially with the rise of smart devices and smartphones. Digital TVs now serve as an ideal environment for multimodal interaction experiences. However, programming these experiences presents significant challenges, particularly when dealing with specific programming languages like NCL (Nested Context Language). NCL 4.0, as part of the TV 3.0 Project, offers promising support for multi-user identification and interaction in digital TV applications, aiming to create a more personalized and participatory digital TV environment. Despite its potential, developing NCL 4.0 applications for multimodal interaction remains complex due to the need for understanding its syntax and semantics, as well as creating applications capable of recognizing and responding to voice commands and differentiating viewers. This paper presents an authoring tool designed to simplify the creation of multimodal interaction applications in NCL 4.0. The tool, implemented as a VSCode extension, automates the process of converting remote control interactions into multimodal links, thus reducing the authoring effort. By integrating voice, gesture, and facial expression interactions, the tool allows developers to focus more on enhancing user experience rather than technical implementation details. The paper details the functionalities and implementation of the tool, demonstrating how it can streamline the development process and improve the flexibility and usability of NCL 4.0 applications. Future work includes further testing and exploration of additional interaction modalities to enhance the tool’s adaptability and inclusiveness.

Palavras-chave: Ferramenta de Autoria, Interação Multimodal, NCL, TV Digital, VSCode

Referências

ABNT. 2021. Digital terrestrial television - Data coding and transmission specification for digital broadcasting - Part 2: Ginga-NCL for fixed and mobile receivers XML application language for application coding (In portuguese).

Fábio Barreto, Raphael S de Abreu, Eyre Brasil B Montevecchi, Marina IP Josué, Pedro Alves Valentim, and Débora C Muchaluat-Saade. 2020. Extending GingaNCL to Specify Multimodal Interactions With Multiple Users (in Portuguese). In Proceedings of the Brazilian Symposium on Multimedia and the Web. 281–288.

Gavin Bierman, Martín Abadi, and Mads Torgersen. 2014. Understanding typescript. In ECOOP 2014–Object-Oriented Programming: 28th European Conference, Uppsala, Sweden, July 28–August 1, 2014. Proceedings 28. Springer, 257–281.

Marco A Casanova, Luiz Tucherman, Maria Julia D Lima, José L Rangel Netto, Noemi Rodriquez, and Luiz FG Soares. 1991. The nested context model for hyperdocuments. In Proceedings of the third annual ACM conference on Hypertext. 193–201.

Eyre Brasil B Montevecchi, Marina IP Josué, Fábio Barreto, Raphael S de Abreu, and Débora C Muchaluat-Saade. 2020. Providing Eye Gaze Interaction for GingaNCL Applications (in Portuguese). In Proceedings of the Brazilian Symposium on Multimedia and the Web. 297–303.

D. C. Muchaluat-Saade and L. F. G. Soares. 2002. XConnector & XTemplate: Improving the Expressiveness and Reuse in Web Authoring Languages. The New Review of Hypermedia and Multimedia Journal 8, 1 (2002), 139–169.

Luiz Fernando Gomes Soares and Rogério Ferreira Rodrigues. 2005. Nested context model 3.0: Part 1–ncm core. Technical Report of the Informatics Department of PUC-Rio 12 (2005).

Luiz Fernando Gomes Soares, Rogério Ferreira Rodrigues, and Márcio Ferreira Moreno. 2007. Ginga-NCL: the declarative environment of the Brazilian digital TV system. Journal of the Brazilian Computer Society 12, 4 (2007), 37–46.

Luiz Fernando G Soares, Rogério F Rodrigues, and Débora C Muchaluat-Saade. 2000. Modeling, authoring and formatting hypermedia documents in the HyperProp system. Multimedia systems 8, 2 (2000), 118–134.

International Telecommunication Union. 2009. ITU-R H761: Nested Context Language (NCL) and Ginga-NCL for IPTV Services. (April 2009).