Low-Cost Training of Image-to-Image Diffusion Models with Incremental Learning and Task/Domain Adaptation

被引:0
|
作者
Antona, Hector [1 ]
Otero, Beatriz [1 ]
Tous, Ruben [1 ]
机构
[1] Univ Politecn Cataluna, Dept Comp Architecture, Jordi Girona 1-3, Barcelona 08034, Spain
关键词
diffusion probabilistic models; deep learning; adaptive learning; transfer learning; image inpainting; image colorization; image-to-image translation; training efficiency;
D O I
10.3390/electronics13040722
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Diffusion models specialized in image-to-image translation tasks, like inpainting and colorization, have outperformed the state of the art, yet their computational requirements are exceptionally demanding. This study analyzes different strategies to train image-to-image diffusion models in a low-resource setting. The studied strategies include incremental learning and task/domain transfer learning. First, a base model for human face inpainting is trained from scratch with an incremental learning strategy. The resulting model achieves an FID score almost equivalent to that of its batch learning equivalent while significantly reducing the training time. Second, the base model is fine-tuned to perform a different task, image colorization, and, in a different domain, landscape images. The resulting colorization models showcase exceptional performances with a minimal number of training epochs. We examine the impact of different configurations and provide insights into the ability of image-to-image diffusion models for transfer learning across tasks and domains.
引用
收藏
页数:21
相关论文
共 50 条
  • [1] Incremental Learning of Multi-Domain Image-to-Image Translations
    Tan, Daniel Stanley
    Lin, Yong-Xiang
    Hua, Kai-Lung
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2021, 31 (04) : 1526 - 1539
  • [2] Domain Bridge for Unpaired Image-to-Image Translation and Unsupervised Domain Adaptation
    Pizzati, Fabio
    de Charette, Raoul
    Zaccaria, Michela
    Cerri, Pietro
    2020 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2020, : 2979 - 2987
  • [3] Cycle-object consistency for image-to-image domain adaptation
    Lin, Che-Tsung
    Kew, Jie-Long
    Chan, Chee Seng
    Lai, Shang -Hong
    Zach, Christopher
    PATTERN RECOGNITION, 2023, 138
  • [4] Image-to-image domain adaptation for vehicle re-identification
    Zhang, Fukai
    Zhang, Lulu
    Zhang, Haiyan
    Ma, Yongqiang
    MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 82 (26) : 40559 - 40584
  • [5] Semantically Consistent Image-to-Image Translation for Unsupervised Domain Adaptation
    Brehm, Stephan
    Scherer, Sebastian
    Lienhart, Rainer
    ICAART: PROCEEDINGS OF THE 14TH INTERNATIONAL CONFERENCE ON AGENTS AND ARTIFICIAL INTELLIGENCE - VOL 2, 2022, : 131 - 141
  • [6] Image-to-image domain adaptation for vehicle re-identification
    Fukai Zhang
    Lulu Zhang
    Haiyan Zhang
    Yongqiang Ma
    Multimedia Tools and Applications, 2023, 82 : 40559 - 40584
  • [7] Diffusion Models for Cross-Domain Image-to-Image Translation with Paired and Partially Paired Datasets
    Bell, Trisk
    Li, Dan
    2024 IEEE 11TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS, DSAA 2024, 2024, : 38 - 45
  • [8] TriGAN: image-to-image translation for multi-source domain adaptation
    Roy, Subhankar
    Siarohin, Aliaksandr
    Sangineto, Enver
    Sebe, Nicu
    Ricci, Elisa
    MACHINE VISION AND APPLICATIONS, 2021, 32 (01)
  • [9] Injecting-Diffusion: Inject Domain-Independent Contents into Diffusion Models for Unpaired Image-to-Image Translation
    Li, Luying
    Ma, Lizhuang
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 282 - 287
  • [10] TriGAN: image-to-image translation for multi-source domain adaptation
    Subhankar Roy
    Aliaksandr Siarohin
    Enver Sangineto
    Nicu Sebe
    Elisa Ricci
    Machine Vision and Applications, 2021, 32