Skelibras: Uma extensa base de dados de Libras construída com esqueletos 2D

Abstract

The recognition of dynamic signs of sign languages is a difficult task that is starting to become feasible with the use of deep neural networks. However, the absence of large annotated databases makes the training of these classifiers unfeasible. In this work, a database called Skelibras was built, containing 57760 skeleton samples (poses) divided into 6572 classes of dynamic signs of Brazilian Sign Language (Libras). Each sign in Skelibras is made up of sequences of poses of the body and hands. The poses are automatically extracted and indexed from videos from the Corpus de Libras database. To extract and organize these annotated data consistently, a methodology capable of identifying and tracking the poses of each speaker, indexing the subtitles and speakers present in the conversations, and indexing the information between the videos acquired from different points of view for a single conversation with their respective subtitles. We performed experiments on variations of deep neural networks based on convolutional layers, dense layers, and LSTMs units to validate and provide preliminary results in the base generated in this work, thus enabling future comparison with new dynamic signal recognition methods, reaching 88.40 % of accuracy in the best experiment, and about 7.5% in the worst of the experiments. Skelibras is a public database and can be accessed at the URL: https://github.com/luqsthunder/Skelibras.