A Neural Network-based Suture-tension Estimation Method Using Spatio-temporal Features of Visual Information and Robot-state Information for Robot-assisted Surgery

In robot-assisted minimally invasive surgery, there is a risk of skin tissue damage or suture failure at the suture site owing to incomplete tension. To avoid these problems and improve the inaccuracy of tension prediction, this study proposes a suture-tension prediction method using spatio-temporal features that simultaneously utilizes visual information obtained from surgical suture images and robot state changes over time. The proposed method can assist in minimally invasive robotic surgical techniques by predicting suture-tension through a neural network with image and robot information as inputs, without additional equipment. The neural network structure of the proposed method was reconstructed using ShuffleNet V2plus and spatio-temporal long-short-term memory, which are suitable for tension prediction. To validate the constructed neural network, we performed suturing expferiments using biological tissue and created a training database. We trained the proposed model using the built database and found that the estimated suture-tension values were similar to the actual tension values. We also found that the estimated tension values performed better than those of the other neural network models.