A New Approach to Orthopedic Surgery Planning Using Deep Reinforcement Learning and Simulation

Computer-assisted orthopedic interventions require surgery planning based on patient-specific three-dimensional anatomical models. The state of the art has addressed the automation of this planning process either through mathematical optimization or supervised learning, the former requiring a handcrafted objective function and the latter sufficient training data. In this paper, we propose a completely model-free and automatic surgery planning approach for femoral osteotomies based on Deep Reinforcement Learning which is capable of generating clinical-grade solutions without needing patient data for training. One of our key contributions is that we solve the real-world task in a simulation environment tailored to orthopedic interventions based on an analytical representation of real patient data, in order to overcome convergence, noise, and dimensionality problems. An agent was trained on simulated anatomy based on Proximal Policy Optimization and inference was performed on real patient data. A qualitative evaluation with expert surgeons and a complementary quantitative analysis demonstrated that our approach was capable of generating clinical-grade planning solutions from unseen data of eleven patient cases. In eight cases, a direct comparison to clinical gold standard (GS) planning solutions was performed, showing our approach to perform equally good or better in 80% (surgeon 1) respectively 100% (surgeon 2) of the cases.