Unsupervised Medical Image Translation With Adversarial Diffusion Models

被引:131
作者
Ozbey, Muzaffer [1 ,2 ]
Dalmaz, Onat [1 ,2 ]
Dar, Salman U. H. [1 ,2 ]
Bedel, Hasan A. [1 ,2 ]
Ozturk, Saban [1 ,2 ,3 ]
Gungor, Alper [1 ,2 ,4 ]
Cukur, Tolga [1 ,2 ]
机构
[1] Bilkent Univ, Dept Elect & Elect Engn, TR-06800 Ankara, Turkiye
[2] Bilkent Univ, Natl Magnet Resonance Res Ctr UMRAM, TR-06800 Ankara, Turkiye
[3] Amasya Univ, Dept Elect Elect Engn, TR-05100 Amasya, Turkiye
[4] SELSAN Res Ctr, TR-06200 Ankara, Turkiye
关键词
Biological system modeling; Computational modeling; Training; Generative adversarial networks; Image synthesis; Task analysis; Generators; Medical image translation; synthesis; unsupervised; unpaired; adversarial; diffusion; generative; ESTIMATING CT IMAGE; RANDOM FOREST; MODALITY; MR;
D O I
10.1109/TMI.2023.3290149
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Imputation of missing images via source-to-target modality translation can improve diversity in medical imaging protocols. A pervasive approach for synthesizing target images involves one-shot mapping through generative adversarial networks (GAN). Yet, GAN models that implicitly characterize the image distribution can suffer from limited sample fidelity. Here, we propose a novel method based on adversarial diffusion modeling, SynDiff, for improved performance in medical image translation. To capture a direct correlate of the image distribution, SynDiff leverages a conditional diffusion process that progressively maps noise and source images onto the target image. For fast and accurate image sampling during inference, large diffusion steps are taken with adversarial projections in the reverse diffusion direction. To enable training on unpaired datasets, a cycle-consistent architecture is devised with coupled diffusive and non-diffusive modules that bilaterally translate between two modalities. Extensive assessments are reported on the utility of SynDiff against competing GAN and diffusion models in multi-contrast MRI and MRI-CT translation. Our demonstrations indicate that SynDiff offers quantitatively and qualitatively superior performance against competing baselines.
引用
收藏
页码:3524 / 3539
页数:16
相关论文
共 84 条
[21]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778
[22]   Cross-Modality Image Synthesis from Unpaired Data Using CycleGAN Effects of Gradient Consistency Loss and Training Data Size [J].
Hiasa, Yuta ;
Otake, Yoshito ;
Takao, Masaki ;
Matsuoka, Takumi ;
Takashima, Kazuma ;
Carass, Aaron ;
Prince, Jerry L. ;
Sugano, Nobuhiko ;
Sato, Yoshinobu .
SIMULATION AND SYNTHESIS IN MEDICAL IMAGING, 2018, 11037 :31-41
[23]   Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network [J].
Hien Van Nguyen ;
Zhou, Kevin ;
Vemulapalli, Raviteja .
MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION - MICCAI 2015, PT I, 2015, 9349 :677-684
[24]  
Ho J., 2020, ADV NEURAL INFORM PR, V33, P6840, DOI DOI 10.48550/ARXIV.2006.11239
[25]   Multimodal Unsupervised Image-to-Image Translation [J].
Huang, Xun ;
Liu, Ming-Yu ;
Belongie, Serge ;
Kautz, Jan .
COMPUTER VISION - ECCV 2018, PT III, 2018, 11207 :179-196
[26]   Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning [J].
Huang, Yawen ;
Shao, Ling ;
Frangi, Alejandro F. .
IEEE TRANSACTIONS ON MEDICAL IMAGING, 2018, 37 (03) :815-827
[27]   Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Coding [J].
Huang, Yawen ;
Shao, Ling ;
Frangi, Alejandro F. .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :5787-5796
[28]  
Iglesias JE, 2013, LECT NOTES COMPUT SC, V8149, P631, DOI 10.1007/978-3-642-40811-3_79
[29]   Image-to-Image Translation with Conditional Adversarial Networks [J].
Isola, Phillip ;
Zhu, Jun-Yan ;
Zhou, Tinghui ;
Efros, Alexei A. .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :5967-5976
[30]  
Jalal A, 2021, ADV NEUR IN, V34