Deep Reinforcement Learning-Based Camera Autofocus with Gaussian Process Regression

Autofocus (AF) is a fundamental task in digital image capturing, yet current approaches often exhibit a poor performance in terms of speed or accuracy. In this paper, we propose a deep reinforcement learning (DRL)-based framework for camera AF, by incorporating a pre-trained MobileNet-v2 encoder for image feature extraction and a Gaussian process regression (GPR) module for in-focus prediction. Specially, we formulate AF as a multi-step process, in which the lens movement trajectory is dynamically generated from a DRL-based policy. By leveraging image features extracted from the MobileNet-v2 encoder, the DRL-based policy can efficiently get close to the in-focus position, which enables the focus search in a fast speed, thus overcoming the drawback of slow focusing procedure suffered by traditional search-based AF methods. A GPR-based in-focus predictor is further designed to terminate the DRL's exploration once it is able to provide a reliable prediction on the in-focus position, by exploiting the sharpness metric values of image patches captured at the DRL-generated trajectory of lens positions. In further contrast with the one-step prediction-based methods, we leverage the sequential information brought additionally by the multi-step exploration, enabling to gain a higher prediction accuracy. Experimental results demonstrate that our approach provides a significant improvement compared to both the search-based and prediction-based AF baselines.