Colombian Dialect Recognition from Call-Center Conversations Using Fusion Strategies

Automatic dialect recognition is a challenging problem with many applications in different fields. Particularly, in customer service, it is used to improve the interaction between customers and providers or to segment offers such that products and services are exposed to the group of greatest interest. This study proposes a bi-modal analysis to discriminate between two Colombian dialects ("Antioqueno" and "Bogotano") using text and speech signals generated in real call-center conversations. First, we evaluated uni-modal strategies considering classical and deep approaches to analyze speech recordings and their corresponding transliterations (text). Then, different fusion strategies were considered to combine the information from speech and text in different stages of the methodology. In early fusion, the uni-modal feature vectors are concatenated and used as input to generate a new model. In late fusion, the scores resulting from the uni-modal classifications are concatenated to form the features vectors to train a support vector machine to predict the dialect. Furthermore, a joint fusion strategy was tested to enhance the characterization process of one mode based on the information from the other one. The results indicate that bi-modal approaches using the late fusion strategy outperform uni-modal approaches by up to 9%. To the best of our knowledge, this is the first work of Colombian dialect recognition based on call-center conversations using a bi-modal approach. Future experiments will consider other fusion strategies where information from text and speech are synchronously merged to improve the dialect classification.