Speech enabled Integrated AR-based Multimodal Language Translation

With rapidly increasing smartphone user base in the country, many opportunities revolving the same can be explored. India being a multilingual nation, domestic/international tourist needs several language solutions on their fingertips. Applications such as instant language translation, maps, travel assistance, routes, restaurants, banks etc. have become a necessity on smartphones in the current scenario. We present a speech enabled augmented reality (AR) smartphones application for Hindi to English translation and vice-versa. With application interface user can move their smartphone over the word, phrase, billboard, signboard, hoarding, milestone etc. to get translated text. Using augmented reality technology the translated text seamlessly overlay [1] to the original text in real time video stream. This is achieved by critical mix of underlying technologies such as augmented reality, Optical Character Recognition (OCR), translation of visual data and speech synthesis. User will be able to effectively engage, communicate and access information originally available in native language.