Feature pyramid self-attention network for respiratory motion prediction in ultrasound image guided surgery

Purpose The robot-assisted automated puncture system under ultrasound guidance can well improve the puncture accuracy in ablation surgery. The automated puncture system requires advanced definition of the puncture location, while the displacement of thoracic-abdominal tumors caused by respiratory motion makes it difficult for the system to locate the best puncture position. Predicting tumor motion is an effective way to help the automated puncture system output a more accurate puncture position. Methods In this paper, we propose a self-attention-based feature pyramid algorithm FPSANet for time-series forecasting, which can extract both linear and nonlinear dependencies of time series. Firstly, we use the temporal convolutional network as the backbone to extract different scale time-series features, and the self-attention module is followed to weigh more significant features to improve nonlinear prediction. Secondly, we use autoregressive models to perform linear prediction. Finally, we directly combine the above two kinds of predictions as the final prediction. Results FPSANet is trained and tested on our private datasets captured from clinical individuals, and we predict the target position after 50 ms, 150 ms, 300 ms and 400 ms. The result shows the evaluation criteria of the MAE is less than 1 mm at 50 ms and 150 ms, and less than 2 mm at 300 ms. Compared with the AR model, bidirectional LSTM and RVM, our method not only outperforms both models in accuracy (AR: similar to 7.7%; bidirectional LSTM: similar to 75.9%; RVM: similar to 76.5%) but is also more stable on different types of respiratory curves. Conclusion Respiratory motion in the liver in actual clinical practice vary widely from person to person, while sometimes having less distinct periodic patterns. Under these conditions, our algorithm has the advantage of excellent stability for prediction on various sequences, and its running time of performing single sequence prediction can meet clinical requirements.