Towards markerless computer-aided surgery combining deep segmentation and geometric pose estimation: application in total knee arthroplasty

Total knee arthroplasty (TKA) is a surgical procedure performed in patients suffering from knee arthritis. The correct positioning of the implants is strongly related to multiple surgical variables that have a tremendous impact on the success of the surgery. Computer-based navigation systems have been investigated and developed in order to assist the surgeon in accurately controlling those surgical variables. The existing technologies are very costly, require additional bone incisions for fixing markers to be tracked, and these markers are usually bulky, interfering with the standard surgical flow. This work presents a markerless navigation system that supports the surgeon in accurately performing the TKA procedure. The proposed system uses a mobile RGB-D camera for replacing the existing optical tracking systems and does not require markers to be tracked. We combine an effective deep learning-based approach for accurately segmenting the bone surface with a robust geometry-based algorithm for registering the bones with pre-operative models. The favourable performance of our pipeline is achieved by (1) employing a semi-supervised labelling approach for generating training data from real TKA surgery data, (2) using effective data augmentation techniques for improving the generalisation capability and (3) using appropriate depth data cleaning strategies. The construction of this complete markerless registration prototype that generalises for unseen intra-operative data is non-obvious, and relevant insights and future research directions can be derived. The experimental results show encouraging performance for video-based TKA.