Real-time Style Transfer for Videos to Enhance the Realism of Simulation of Laparoscopic Surgeries

Surgical simulation has repeatedly proven its potential, but is limited by the lack of realism to the surgical experience. For laparoscopic surgery simulation, an image processing method known as style transfer can improve the realism by transferring the style of a picture from surgery onto the video of the simulator using a stylisation network. In this article, we propose an adjustable style transfer algorithm for videos to improve the realism of silicone based models. The results show that our method can successfully implement multiple stylisations in realtime onto the video of the simulator while maintaining temporal consistency and overall smoothness of the video. Comparing to other style-transfer methods, our technique can offer multiple stylisations and, except from the method of Johnson et al. [6], achieves better realness score when evaluated by surgeons. This method can also be trained on a generic database in only 3h43 contrary to the other image processing techniques such as image translation which require specific training datasets.