A CNN-LSTM based ensemble framework for in-air handwritten Assamese character recognition

In-air handwriting is a contemporary human computer interaction (HCI) technique which enables users to write and communicate in free space in a simple and intuitive manner. Air-written characters exhibit wide variations depending upon different writing styles of users and their speed of articulation, which presents a great challenge towards effective recognition of linguistic characters. So, in this paper we have proposed an ensemble model for in-air handwriting recognition which is based on convolutional neural network (CNN) and a long short-term memory neural network (LSTM-NN). The method collaborates overall character trajectory appearance modeling and temporal trajectory feature modeling for efficient recognition of varied types of air-written characters. In contrast to two-dimensional handwriting, in-air handwriting generally involves writing of characters interlinked by a continuous stroke, which makes segregation of intended writing activity from insignificant connecting motions an intricate task. So, a two-stage statistical framework is incorporated in the system for automatic detection and extraction of relevant writing segments from air-written characters. Identification of writing events from a continuous stream of air-written data is accomplished by formulating a Markov Random Field (MRF) model, while the segmentation of writing events into meaningful handwriting segments and redundant parts is performed by implementation of a Mahalanobis distance (MD) classifier. The proposed approach is assessed on an air-written character dataset comprising of Assamese vowels, consonants and numerals. The experimental results connote that our hybrid network can assimilate more information from the air-writing patterns and hence offer better recognition performance than the state-of-the-art approaches.