Lightweight Deep Neural Network for Articulated Joint Detection of Surgical Instrument in Minimally Invasive Surgical Robot

Vision-based detection and tracking of surgical instrument are attractive because it relies purely on surgical instrument already in the operating scenario. The vision knowledge of the surgical instruments is a crucial piece of topic for surgical task understanding, autonomous robot control and human–robot collaborative surgeries to enhance surgical outcomes. In this work, a novel method has been demonstrated by developing a multitask lightweight deep neural network framework to explore surgical instrument articulated joint detection. The model has an end-to-end architecture with two branches, which share the same high-level visual features provided by a lightweight backbone while holding respective layers targeting for specific tasks. We have designed a novel subnetwork with joint detection branch and an instrument classification branch to sufficiently take advantage of the relatedness of surgical instrument presence detection and surgical instrument articulated joint detection tasks. The lightweight joint detection branch has been employed to efficiently locate the articulated joint position with simultaneously holding low computational cost. Moreover, the surgical instrument classification branch is introduced to boost the performance of joint detection. The two branches are merged to output the articulated joint location with respective instrument type. Extensive validation has been conducted to evaluate the proposed method. The results demonstrate promising performance of our proposed method. The work represents the feasibility to perform real-time surgical instrument articulated joint detection by taking advantage of the components of surgical robot system, contributing to the reference for further surgical intelligence.