Methodology based on Computer Vision and Machine Learning to guide the Design of Augmentative and Alternative Communication Systems using Personalized Gestural Interaction

• Rúbia E. O. Schultz Ascari UTFPR
• Luciano Silva UFPR
• Roberto Pereira UFPR

Resumo

People who have motor disabilities associated with some difficulty in speech demand alternative means to interact with other people and the environment in which they are inserted. Augmentative and Alternative Communication (AAC) refers to all communication forms that can complement or replace speech. The practice of AAC mediated by computational applications represents a very attractive alternative. For people with motor and speech difficulties, gestural interaction can be a way to make interaction with AAC systems feasible. In this thesis, a methodology called MyPGI (Methodology to yield Personalized Gestural Interaction) was developed and validated to guide the design of AAC systems for people with motor and speech difficulties in order to promote greater autonomy for them in handling computer systems and for monitoring by their caregivers. The methodology uses Computer Vision and Machine Learning techniques to enable non-invasive and personalized gestural interaction using low-cost devices. MyPGI was applied and evaluated in real case studies with people with disabilities, informing the design of an interactive system named PGCA (Personal Gesture Communication Assistant) to allow the creation and use of personalized gestural languages for people with severe motor and speech difficulties. This thesis, defended at the Federal University of Parana’s Graduate Program in Informatics, shows conceptual, methodological, and technical contributions, with publications in high-level vehicles, registered software, and social technology available for free and open use.

Palavras-chave: Assistive Technology, Augmentative and Alternative Communication, Computer Vision, Gesture Recognition, Accessibility