Displacement detection with sub-pixel accuracy and high spatial resolution using deep learning

Purpose The purpose of this study was to detect two dimensional and sub-pixel displacement with high spatial resolution using an ultrasonic diagnostic apparatus. Conventional displacement detection methods assume neighborhood uniformity and cannot achieve both high spatial resolution and sub-pixel displacement detection. Methods A deep-learning network that utilizes ultrasound images and output displacement distribution was developed. The network structure was constructed by modifying FlowNet2, a widely used network for optical flow estimation, and a training dataset was developed using ultrasound image simulation. Detection accuracy and spatial resolution were evaluated via simulated ultrasound images, and the clinical usefulness was evaluated with ultrasound images of the liver exposed to high-intensity-focused ultrasound (HIFU). These results were compared to the Lucas-Kanade method, a conventional sub-pixel displacement detection method. Results For a displacement within +/- 40 mu m (+/- 0.6 pixels), a pixel size of 67 mu m, and signal noise of 1%, the accuracy was above 0.5 mu m and 0.2 mu m, the precision was above 0.4 mu m and 0.3 mu m, and the spatial resolution was 1.1 mm and 0.8 mm for the lateral and axial displacements, respectively. These improvements were also observed in the experimental data. Visualization of the lateral displacement distribution, which determines the edge of the treated lesion using HIFU, was also realized. Conclusion Two-dimensional and sub-pixel displacement detection with high spatial resolution was realized using a deep-learning methodology. The proposed method enabled the monitoring of small and local tissue deformations induced by HIFU exposure.