A Brazilian Sign Language Video Database for Automatic Recognition

• Priscila Gameiro CEFET-RJ
• Wesley Passos UFRJ
• Gabriel Araujo CEFET-RJ
• Amaro de Lima CEFET-RJ
• Jonathan Gois CEFET-RJ
• Anna Corbo CEFET-RJ

Abstract

Communication is a basic human necessity, but the deaf community is still facing some barriers. In Brazil, there are about 9.7 million people with some degree of hearing impairment but most Brazilians do not know the Brazilian Sign Language (Libras). Wearable gadgets like gloves or bracelets with sensors are a good tool to help in communication, but these solutions are pricey or intrusive. On the other hand, computer vision solutions are cheaper but challenging mainly due to the absence of reliable video databases. In this work, we propose a video database compounded by different signs, with various backgrounds, and different speakers. We also developed a system to perform computer vision-based sign recognition using the k-nearest neighbors (K-NN) and random forest (RF). In our tests, we achieved an average accuracy of 65.81% in multiclass classification.

Keywords: Databases, Gesture recognition, Assistive technology, Video sequences, Radio frequency, Feature extraction, Task analysis, Sign Language, LIBRAS, Computer Vision, Gait Energy Image, Image Processing