Depth Estimation of Optically Transparent Microrobots Using Convolutional and Recurrent Neural Networks

Estimating the three-dimensional (3D) position of microrobots is necessary in order to develop closed-loop control techniques and to improve the user's 3D perception in the micro-scale. This paper describes a depth estimation method based on supervised learning for optically transparent microrobots of known geometry. The proposed methodology uses Convolutional Neural Networks (CNNs) combined with a Recurrent Network, in particular a Long Short-Term Memory (LSTM) cell for depth regression. The proposed depth regression model is independent of the 3D orientation of the microrobot and is robust to varying illumination levels while it uses learned data-specific features. The model is trained and validated using microscope images and ground truth data generated from 3D-printed microrobots imaged in an Optical Tweezers (OT) setup. The validation results demonstrate that the proposed trained model can reconstruct the depth of the microrobot independently of its 3D orientation with submicron accuracy for the test set.