Rúbia E. O. Schultz Ascari
Roberto Pereira
ABSTRACT
People who have motor disabilities associated with some difficulty in speech demand alternative means to interact with other people and the environment in which they are inserted. Augmentative and Alternative Communication (AAC) refers to all communication forms that can complement or replace speech. The practice of AAC mediated by computational applications represents a very attractive alternative. For people with motor and speech difficulties, gestural interaction can be a way to make interaction with AAC systems feasible. In this thesis, a methodology called MyPGI (Methodology to yield Personalized Gestural Interaction) was developed and validated to guide the design of AAC systems for people with motor and speech difficulties in order to promote greater autonomy for them in handling computer systems and for monitoring by their caregivers. The methodology uses Computer Vision and Machine Learning techniques to enable non-invasive and personalized gestural interaction using low-cost devices. MyPGI was applied and evaluated in real case studies with people with disabilities, informing the design of an interactive system named PGCA (Personal Gesture Communication Assistant) to allow the creation and use of personalized gestural languages for people with severe motor and speech difficulties. This thesis, defended at the Federal University of Parana's Graduate Program in Informatics, shows conceptual, methodological, and technical contributions, with publications in high-level vehicles, registered software, and social technology available for free and open use.
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, et al. 2016. Tensor-flow: a system for large-scale machine learning.. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Vol. 16. 265--283.Google Scholar
- Ethem Alpaydin. 2020. Introduction to machine learning. MIT Press.Google ScholarDigital Library
- Rúbia E O Schultz Ascari, Roberto Pereira, and Luciano Silva. 2018. Mobile Interaction for Augmentative and Alternative Communication: a Systematic Mapping. SBC Journal on 3D Interactive Systems 9, 2 (2018), 105--118.Google Scholar
- Rúbia E O Schultz Ascari, Roberto Pereira, and Luciano Silva. 2018. Towards a Methodology to Support Augmentative and Alternative Communication by means of Personalized Gestural Interaction. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems (IHC 2018). ACM, 1--12.Google ScholarDigital Library
- Rúbia E O Schultz Ascari, Roberto Pereira, and Luciano Silva. 2020. Computer vision-based methodology to improve interaction for people with motor and speech impairment. ACM Transactions on Accessible Computing (TACCESS) 1, 1 (2020), 1--33.Google Scholar
- Rúbia E O Schultz Ascari, Roberto Pereira, and Luciano Silva. 2020. Personalized Gestural Interaction Applied in a Gesture Interactive Game-based Approach for Disabled People. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI 2020). ACM, 1--11.Google Scholar
- Rúbia E O Schultz Ascari, Roberto Pereira, and Luciano Silva. 2021. Computer vision applied to improve interaction and communication of people with motor disabilities: A systematic mapping. Technology and Disability 33, 1 (2021), 1--18.Google Scholar
- Rúbia E O Schultz Ascari, Luciano Silva, and Roberto Pereira. 2019. Personalized Interactive Gesture Recognition Assistive Technology. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (IHC 2019). ACM, 1--12.Google ScholarDigital Library
- M. Cecilia Calani Baranauskas, Clarisse S. de Souza, and Roberto Pereira. 2015. GranDIHC-BR-Grand Research Challenges for Human-Computer Interaction in Brazil. Human-Computer Interaction Special Committee (CEIHC) of the Brazilian Computer Society (SBC) (2015).Google Scholar
- Samit Bhattacharya, Debasis Samanta, and Anupam Basu. 2008. Performance models for automatic evaluation of virtual scanning keyboards. IEEE Transactions on Neural Systems and Rehabilitation Engineering 16, 5 (2008), 510--519.Google ScholarCross Ref
- Chi-hau Chen. 2015. Handbook of Pattern Recognition and Computer Vision. World Scientific.Google Scholar
- Louis Cohen, Lawrence Manion, and Keith Morrison. 2000. Research methods in education [5 th edn] London: Routledge Falmer. Teaching in higher education 41 (2000), 21.Google Scholar
- Wikimedia Commons. 2017. CommonsWikimedia. https://commons.wikimedia.org/wiki/File:Turtle_clip_art.svg. Acessado em 12/05/2020.Google Scholar
- Sibanjan Das. 2016. Classification methods. In Data Science Using Oracle Data Miner and Oracle R Enterprise. Springer, 189--237.Google Scholar
- Sky Box-Sunny Day. 2015. OpenGameArt.org. https://opengameart.org. Acessado em 12/05/2020.Google Scholar
- Bradley Efron. 1992. Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics. Springer, 569--593.Google Scholar
- S Yu Eroshkin, NA Kameneva, DV Kovkov, and AI Sukhorukov. 2017. Conceptual system in the modern information management. Procedia Computer Science 103 (2017), 609--612.Google ScholarDigital Library
- Ivo Everts, Nicu Sebe, Graeme A Jones, et al. 2007. Cooperative Object Tracking with Multiple PTZ Cameras. In International Conference on Image Analysis and Processing (ICIAP 2007), Vol. 7. 323--330.Google Scholar
- Seymour Geisser. 1975. The predictive sample reuse method with applications. J. Amer. Statist. Assoc. 70, 350 (1975), 320--328.Google ScholarCross Ref
- Cindy Gevarter et al. 2013. Comparisons of intervention components within augmentative and alternative communication systems for individuals with developmental disabilities: A review of the literature. Research in developmental disabilities 34, 12 (2013), 4404--4414.Google Scholar
- Google. 2019. Google Image. http://images.google.com. Acessado em 12/11/2019.Google Scholar
- Shaun K Kane, Amy Hurst, Erin Buehler, Patrick A Carrington, and Michele A Williams. 2014. Collaboratively designing assistive technology. Interactions 21, 2 (2014), 78--81.Google ScholarDigital Library
- Ron Kohavi. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of Fourteenth International Joint Conference on Artificial Intelligence (IJCAI), Vol. 14. Montreal, Canada, 1137--1145.Google Scholar
- Denis Lalanne, Laurence Nigay, Peter Robinson, et al. 2009. Fusion engines for multimodal input: a survey. In Proceedings of the 2009 International Conference on Multimodal Interfaces. ACM, 153--160.Google ScholarDigital Library
- Larry Laudan. 1978. Progress and its problems: Towards a theory of scientific growth. Vol. 282. Univ of California Press.Google Scholar
- Bruce D Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence. Vancouver, BC, Canada.Google ScholarDigital Library
- Sharynne McLeod. 2004. Speech pathologists' application of the ICF to children with speech impairment. Advances in Speech Language Pathology 6, 1 (2004), 75--81.Google ScholarCross Ref
- Laura Montanini, Enea Cippitelli, Ennio Gambi, and Susanna Spinsante. 2015. Low complexity head tracking on portable android devices for real time message composition. Journal on Multimodal User Interfaces 9, 2 (2015), 141--151.Google ScholarCross Ref
- Johann Mouton. 1996. Understanding social research. Van Schaik Publishers.Google Scholar
- Hoang D Nguyen and Danny Chiang Choon Poo. 2017. Unified Structured Framework for mHealth Analytics: Building an Open and Collaborative Community. In International Conference on Social Computing and Social Media. Springer, 440--450.Google ScholarCross Ref
- Antti Oulasvirta and Kasper Hornbæk. 2016. HCI research as problem-solving. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4956--4967.Google ScholarDigital Library
- Roger Pressman and Bruce Maxim. 2016. Engenharia de Software - 8a Edição. McGraw Hill Brasil.Google Scholar
- Olga Russakovsky, Jia Deng, Hao Su, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252.Google ScholarDigital Library
- Sancho Salcedo-Sanz, José Luis Rojo-Álvarez, Manel Martínez-Ramón, and Gustavo Camps-Valls. 2014. Support vector machines in engineering: an overview. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 4, 3 (2014), 234--267.Google ScholarDigital Library
- Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818--2826.Google ScholarCross Ref
- Martin A Tanner and Wing Hung Wong. 1987. The calculation of posterior distributions by data augmentation. Journal of the American statistical Association 82, 398 (1987), 528--540.Google ScholarCross Ref
Index Terms
- Methodology based on Computer Vision and Machine Learning to guide the Design of Augmentative and Alternative Communication Systems using Personalized Gestural Interaction
Recommendations
Towards a Methodology to Support Augmentative and Alternative Communication by means of Personalized Gestural Interaction
IHC '18: Proceedings of the 17th Brazilian Symposium on Human Factors in Computing SystemsAugmentative and Alternative Communication (AAC) involves the use of non-verbal modes as a complement or substitute for spoken language, supporting communicative abilities of people, especially people with speech limitations. Computing systems have been ...
Personalized gestural interaction applied in a gesture interactive game-based approach for people with disabilities
IUI '20: Proceedings of the 25th International Conference on Intelligent User InterfacesTechnology can support people with disabilities to participate in social and economic life. Using relevant Human-Computer Interaction, as obtained through Intelligent User Interfaces, people with motor and speech impairments may be able to communicate ...
Computer Vision-based Methodology to Improve Interaction for People with Motor and Speech Impairment
Augmentative and Alternative Communication (AAC) aims to complement or replace spoken language to compensate for expression difficulties faced by people with speech impairments. Computing systems have been developed to support AAC; however, partially ...
Comments