Improving Pulmonary CT Image Generation with Transformer-based Generative Adversarial Networks

Computed Tomography (CT) imaging serves as a crucial component in modern medical diagnostics by providing important information about internal structures of human bodies. Unfortunately, CT often faces the problem of data scarcity, due to radiation exposure, the need for skilled professionals and data privacy concerns. Therefore, generative models, such as Generative Adversarial Networks (GANs),1 have been widely applied to generating synthetic CT images, largely changing various aspects of medical image generation and analysis. However, directly applying GANs to CT image generation remains challenging.2 In particular, several representative GAN-based models, including Deep Convolutional GANs (DCGAN)3 and bigGANs,4 cannot directly generate large 3D volumes of CT scans. One important reason is that the consistency and dependency between CT scans are not appropriately handled by those GAN-based models. To model 3D CT scans, large volumes of CT images are treated in similar terms to time series. However, GAN models built up on Recurrent Neural Networks (RNNs) are not able to characterize long sequences of data due to training difficulties. In this paper, we propose Transformer-based GAN models5 to capture long sequences of CT scans. We conduct experiments on the LUNA16 pulmonary CT image dataset to verify the proposed methods. The empirical results demonstrate that the proposed models are able to successfully generate large CT volumes with hundreds of CT slices. The authors confirm that this work has not been submitted before.