A Crowdsourcing Method for Sign Segmentation in Brazilian Sign Language Videos

• Marcello Novaes de Amorim UFES
• Celso A. S. Santos UFES
• Orivaldo de L. Tavares UFES

Resumo

Like the spoken languages, sign languages are not universal and vary in different countries. LIBRAS (Brazilian Sign Language) is the second official language of Brazil and it is the language adopted by Brazilian Deaf's community to communicate. The signs of LIBRAS are composed of hand configurations, facial expressions and are affected by space and intensity modifiers, which makes their recognition more complicated than the simple identification of hand signs. The signs are arranged, according to a grammar, respecting form phrases, clauses, and sentences like any other spoken or sign language. The automatic machine translation of a sign language typically includes an initial phase for detecting sign boundaries. In this paper, we apply a crowdsourcing method to identifying signs boundaries present in pre-recorded videos those features LIBRAS interpreters. The limits or boundaries of the signs in the videos were established from the processing of contributions from workers from different countries, who have supposedly never heard of LIBRAS nor any other sign languages. To evaluate the segmentation process, we compared the sign boundaries identified by the crowd with the ground truth provided by a team of LIBRAS experts, who also assessed the quality of the delimitation of the identified signs. Our analysis showed that our crowdsourcing method was able to get 93.75% of the sign boundaries successfully.

Palavras-chave: Crowdsourcing, segmentation, translation video, LIBRAS