Recent advances in deep learning models: a systematic literature review

In recent years, deep learning has evolved as a rapidly growing and stimulating field of machine learning and has redefined state-of-the-art performances in a variety of applications. There are multiple deep learning models that have distinct architectures and capabilities. Up to the present, a large number of novel variants of these baseline deep learning models is proposed to address the shortcomings of the existing baseline models. This paper provides a comprehensive review of one hundred seven novel variants of six baseline deep learning models viz. Convolutional Neural Network, Recurrent Neural Network, Long Short Term Memory, Generative Adversarial Network, Autoencoder and Transformer Neural Network. The current review thoroughly examines the novel variants of each of the six baseline models to identify the advancements adopted by them to address one or more limitations of the respective baseline model. It is achieved by critically reviewing the novel variants based on their improved approach. It further provides the merits and demerits of incorporating the advancements in novel variants compared to the baseline deep learning model. Additionally, it reports the domain, datasets and performance measures exploited by the novel variants to make an overall judgment in terms of the improvements. This is because the performance of the deep learning models are subject to the application domain, type of datasets and may also vary on different performance measures. The critical findings of the review would facilitate the researchers and practitioners with the most recent progressions and advancements in the baseline deep learning models and guide them in selecting an appropriate novel variant of the baseline to solve deep learning based tasks in a similar setting.