AUPOD: End-to-End Automatic Poster Design by Self-Supervision

被引:3
作者
Huang, Dongjin [1 ]
Li, Jinyao [1 ]
Liu, Chuanman [1 ]
Liu, Jinhua [1 ]
机构
[1] Shanghai Univ, Shanghai Film Acad, Shanghai 200072, Peoples R China
关键词
Layout; Feature extraction; Training; Pipelines; Task analysis; Neural networks; Visualization; Design automation; design aesthetic; artificial intelligence; neural networks; machine learning; SALIENT OBJECT;
D O I
10.1109/ACCESS.2022.3171033
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The automatic design has become a popular topic in the application field of computer vision technologies. Previous methods for automatic design are mostly saliency-based, relying on an off-the-shelf model for saliency map detection and hand-crafted aesthetic rules for ranking on multiple proposals. We argue that the multi-stage generation and the excessive reliance on saliency map hindered the progress of pursuing better automatic design solutions. In this work, we explore the possibility of a saliency-free solution in a representative scenario, automatic poster design. We propose a novel end-to-end framework to solve the automatic poster design problem, which is divided into the layout prediction and attributes identification sub-tasks. We design a neural network based on multi-modality feature extraction to learn the two sub-tasks jointly. We train the deep neural network in our framework with automatically extracted supervision from semi-structured posters, bypassing a large amount of required manual labor. Both qualitative and quantitative results show the impressive performance of our end-to-end approach after discarding the explicit saliency detection module. Our system learned on self-supervision performs well on the automatic design by learning aesthetic constraints implicitly in the neural networks.
引用
收藏
页码:47348 / 47360
页数:13
相关论文
共 54 条
[1]  
[Anonymous], 2017, P CVPR, DOI [DOI 10.1109/CVPR.2017.199, DOI 10.1109/CVPR.2017.472]
[2]   SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J].
Badrinarayanan, Vijay ;
Kendall, Alex ;
Cipolla, Roberto .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2017, 39 (12) :2481-2495
[3]   What is a Salient Object? A Dataset and a Baseline Model for Salient Object Detection [J].
Borji, Ali .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2015, 24 (02) :742-756
[4]   Fast approximate energy minimization via graph cuts [J].
Boykov, Y ;
Veksler, O ;
Zabih, R .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2001, 23 (11) :1222-1239
[5]   Learning Visual Importance for Graphic Designs and Data Visualizations [J].
Bylinskii, Zoya ;
Kim, Nam Wook ;
O'Donovan, Peter ;
Alsheikh, Sami ;
Madan, Spandan ;
Pfister, Hanspeter ;
Durand, Fredo ;
Russell, Bryan ;
Hertzmann, Aaron .
UIST'17: PROCEEDINGS OF THE 30TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY, 2017, :57-69
[6]   Automatic Image Cropping : A Computational Complexity Study [J].
Chen, Jiansheng ;
Bai, Gaocheng ;
Liang, Shaoheng ;
Li, Zhengqin .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :507-515
[7]  
Chen T., 2020, ARXIV
[8]   Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation [J].
Chen, Xiaocong ;
Huang, Chaoran ;
Yao, Lina ;
Wang, Xianzhi ;
Liu, Wei ;
Zhang, Wenjie .
2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
[9]  
Clark K., 2020, P INT C LEARN REPR I
[10]  
Devlin Jacob, 2018, ANN C N AM CHAPTER A