Follow the Curve: Robotic Ultrasound Navigation With Learning-Based Localization of Spinous Processes for Scoliosis Assessment

The scoliosis progression in adolescents requires close monitoring to timely take treatment measures. Ultrasound imaging is a radiation-free alternative in scoliosis assessment to X-ray, which is typically used in clinical practice. However, ultrasound images are prone to speckle noise, making it challenging for sonographers to detect bony features and follow the spinal curvature. This study introduces a novel robotic ultrasound approach for spinous process localization and automatic spinal curvature tracking for scoliosis assessment. The positions of the spinous processes are computed using a fully connected network with a deconvolutional head. A 5-fold cross-validation was performed on a dataset of ultrasound images from 25 human subjects with scoliosis. The resulting percentage of correct keypoints of the spinous process is 0.966 +/- 0.027 with a mean distance error of 1.0 +/- 0.99 mm. We use this machine learning-based method to guide the motion of the robot-held ultrasound probe and to follow the spinal curvature while capturing ultrasound images. We present a new force-driven controller that automatically adjusts the pose and orientation of the probe relative to the skin surface, which ensures a good acoustic coupling between the probe and skin. We extended the network architecture to additionally perform classification of the spine into its regions, i.e., sacrum, lumbar, and thoracic, which are used to adjust the probe's orientation to account for the varying curvature along the spine. After the autonomous scanning, the acquired data is used to reconstruct the coronal spinal image, where the deformity of the scoliosis spine can be assessed and measured. The proposed learning-based method for anatomical landmarks localization was compared to conventional methods based on phase symmetry and image intensity. The learning-based method proved to be more precise for spinous process localization while processing images at a faster rate, which is advantageous for real-time scoliosis scanning. To evaluate the performance of our robotic method, we conducted an experimental study with human scoliosis subjects where deviations of the spinous process from the image center can be compared to those appearing in a manual scan. Our results show that the robotic approach reduces the mean error of spinal curvature following for mild scoliosis from 4.6 +/- 4.6mm (manual scanning) to 1.0 +/- 0.8mm (robotic scanning); For moderate scoliosis from 4.3 +/- 3.9mm (manual scanning) to 2.8 +/- 1.8mm (robotic scanning). The angles of spinal deformity measured on spinal reconstruction images were similar for both methods, implying that they equally reflect human anatomy. The spinal region-specific moment-based probe orientation control showed to improve the scanning performance. An ablation study was performed to investigate the importance of each component of the proposed system.