SimpLex: a lexical text simplification architecture

Text simplification (TS) is the process of generating easy-to-understand sentences from a given sentence or piece of text. The aim of TS is to reduce both the lexical (which refers to vocabulary complexity and meaning) and syntactic (which refers to the sentence structure) complexity of a given text or sentence without the loss of meaning or nuance. In this paper, we present SimpLex, a novel simplification architecture for generating simplified English sentences. To generate a simplified sentence, the proposed architecture uses either word embeddings (i.e., Word2Vec) and perplexity, or sentence transformers (i.e., BERT, RoBERTa, and GPT2) and cosine similarity. The solution is incorporated into a user-friendly and simple-to-use software. We evaluate our system using two metrics, i.e., SARI and Perplexity Decrease. Experimentally, we observe that the transformer models outperform the other models in terms of the SARI score. However, in terms of perplexity, the word embedding-based models achieve the biggest decrease. Thus, the main contributions of this paper are: (1) We propose a new word embedding and transformer-based algorithm for text simplification; (2) we design SimpLex—a modular novel text simplification system—that can provide a baseline for further research; and (3) we perform an in-depth analysis of our solution and compare our results with two state-of-the-art models, i.e., LightLS as reported by Glavaš and Štajner (in: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, 2015) and NTS-w2v as reported by Nisioi et al. (in: Proceedings of the 55th annual meeting of the association for computational linguistics, 2017). We also make the code publicly available online.