TensorRT-based Surgical Instrument Detection Assessment for Deep Learning on Edge Computing

The rise of minimally invasive surgery (MIS) can mainly be attributed to the exponential growth in technology and the evolution of laparoscopic instrumentation over the past two decades. However, advanced technologies in the Operating Theatre, such as imaging systems, robotics, computer assistance, navigation, and monitoring, involve significant effort and a very high level of training and education. Deep Learning has had a major impact on a range of surgical procedures, such as optimizing workflow, surgical training, intraoperative assistance, patient safety, and efficiency. However, it also requires high computational and memory resources. There has been a lot of research into optimizing deep learning models to balance performance and accuracy under limited resources. Techniques like post-training quantization can significantly reduce model size and latency. In this paper, we explore TensorRT-based techniques on edge devices to achieve real-time inference without compromising accuracy under limited compute. For this purpose, a YOLOv5 model is pretrained on M2CAI 2016, to detect and recognize surgical instruments in Laparoscopy and evaluated using mean average precision (mAP) and inference time. We deployed object detection models on an edge device to test real-time performance, exploring the trade-off between accuracy and speed. This paper gives a review looking at how deep learning and edge computing intersect and how to optimize deep learning for edge devices with limited resources.