GAN-Based Medical Image Registration for Augmented Reality Applications

Recently, with the development of AR/VR technology, boosting virtual information to the real world has been applied to various fields to increase convenience. In particular, in the medical field, different types of image information, such as X-ray, CT, and MRI data are used in surgery with AR devices for medical diagnosis and analysis of the cause of a patient's disease. This paper proposes an approach by which to explore a patient's posture and joints to match X-ray image information in an AR environment using deep learning networks such as GAN along with pose estimation. Thereby, we employ the CP-VTON+ virtual try-on network architecture to map chest X-ray information to the patient's body. Finally, we compare the try-on results of the chest X-ray image of the patient's body using the proposed method and CP-VTON+. The mean SSIM value of the proposed method is 0.0272 higher than that of CP-VTON+, and the mean PSNR value is 5.49 higher than that of CP-VTON+. The proposed method is more appropriate for application to AR devices for medical diagnosis and analysis due to the characteristics of medical images, as even minor misdiagnoses can lead to fatalities.