3D Occupancy Reconstruction in Dynamic and Deforming Surgical Environments

Vision dimensionality during minimally invasive surgery is a critical contributor to patient success. Traditional visualizations of the surgical scene are 2D camera streams that obfuscate depth perception inside the abdominal cavity. A lack of depth in surgical views cause surgeons to miss tissue targets, induce blood loss, and incorrectly assess deformation. 3D sensors, while offering key depth information, are expensive and often incompatible with current sterilization techniques. Furthermore, methods inferring a 3D space from stereoscopic video struggle with the inherent lack of unique features in the biological domain. We present an application of deep learning models that can assess simple binary occupancy from a single camera perspective to recreate the surgical scene in high-fidelity. Our quantitative results (IoU=0.82, log loss=0.346) indicate a strong representational capability for structure in surgical scenes, enabling surgeons to reduce patient injury during minimally invasive surgery.