Neural machine translation from text to sign language

The paper describes an ongoing project aimed at developing a neural machine translation approach to translating text into sign language, with the translation result presented by a realistic three-dimensional avatar. The approach can be applied to translate texts, books, and Internet pages, improving access to information for deaf individuals and contributing to the social, educational, and labor inclusion of these citizens. The paper elaborates on four fundamental issues related to the approach: (i) the establishment of a written representation of the sign language; (ii) the construction of a parallel corpus involving text from an oral language and the written representation of sign language; (iii) the establishment of a neural machine translation model to perform the translation; (iv) the visual presentation of the translation by a signing avatar. Although focused on translating Brazilian Portuguese into Brazilian Sign Language, the concepts discussed in the paper are general enough to be applied to other written-signed language pairs.