Infer-HiRes: Accelerating Inference for High-Resolution Images with Quantization and Distributed Deep Learning

High-Resolution Images are being used in various applications, including Medical Imaging, Satellite Imagery, and Surveillance. Due to the evolution of Deep Learning (DL) and its widespread usage, it has also become a prominent choice for high-resolution image applications. But, large image sizes and denser convolutional neural networks pose limitations over computation and memory requirements. To overcome these challenges, several studies have discussed efficient approaches to accelerate training, but the inference of high-resolution images with deep learning and quantization techniques remains unexplored. In this paper, we propose accelerated and memory efficient inference techniques leveraging quantization techniques to reduce the memory and computation requirements while maintaining accuracy. Furthermore, we utilize different parallelism for Distributed DL to enable inference for high-resolution images on out-of-core models. We demonstrate an average 6.5x speedup and 4.55x memory reduction with a single GPU using INT8 quantization. By utilizing Distributed DL, we enabled inference for scaled images, achieving an average 1.58x speedup and 1.57x memory reduction using half-precision. To the best of our knowledge, this paper is the first in the literature to focus on highresolution image inference using quantization with the support of Distributed DL.