Gradient-Boosted Based Structured and Unstructured Learning

被引:0
作者
Gavito, Andrea Trevino [1 ]
Klabjan, Diego [1 ]
Utke, Jean [2 ]
机构
[1] Northwestern Univ, Evanston, IL 60208 USA
[2] Allstate Insurance Co, Northbrook, IL USA
来源
ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2023, PT III | 2023年 / 14256卷
关键词
Deep learning; Multimodal learning; Gradient boosting;
D O I
10.1007/978-3-031-44213-1_37
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose two frameworks to deal with problem settings in which both structured and unstructured data are available. Structured data problems are best solved by traditional machine learning models such as boosting and tree-based algorithms, whereas deep learning has been widely applied to problems dealing with images, text, audio, and other unstructured data sources. However, for the setting in which both structured and unstructured data are accessible, it is not obvious what the best modeling approach is to enhance performance on both data sources simultaneously. Our proposed frameworks allow joint learning on both kinds of data by integrating the paradigms of boosting models and deep neural networks. The first framework, the boosted-feature-vector deep learning network, learns features from the structured data using gradient boosting and combines them with embeddings from unstructured data via a two-branch deep neural network. Secondly, the two-weak-learner boosting framework extends the boosting paradigm to the setting with two input data sources. We present and compare first- and second-order methods of this framework. Our experimental results on both public and real-world datasets show performance gains achieved by the frameworks over selected baselines by magnitudes of 0.1%-4.7%.
引用
收藏
页码:439 / 451
页数:13
相关论文
共 50 条
[11]   Combining structured and unstructured data for predictive models: a deep learning approach [J].
Dongdong Zhang ;
Changchang Yin ;
Jucheng Zeng ;
Xiaohui Yuan ;
Ping Zhang .
BMC Medical Informatics and Decision Making, 20
[12]   Deep Learning Multimodal for Unstructured and Semi-Structured Textual Documents Classification [J].
Katamesh, Nany ;
Abu-Elnasr, Osama ;
Elmougy, Samir .
CMC-COMPUTERS MATERIALS & CONTINUA, 2021, 68 (01) :589-606
[13]   Deep Learning to Predict Hospitalization at Triage: Integration of Structured Data and Unstructured Text [J].
Arnaud, Emilien ;
Elbattah, Mahmoud ;
Gignon, Maxime ;
Dequen, Gilles .
2020 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2020, :4836-4841
[14]   BOOMER - An algorithm for learning gradient boosted multi-label classification rules [J].
Rapp, Michael .
SOFTWARE IMPACTS, 2021, 10
[15]   Gradient Boosted Feature Selection [J].
Xu, Zhixiang ;
Huang, Gao ;
Weinberger, Kilian Q. ;
Zheng, Alice X. .
PROCEEDINGS OF THE 20TH ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING (KDD'14), 2014, :522-531
[16]   DeepSSR: a deep learning system for structured recognition of text images from unstructured paper-based medical reports [J].
Liu, Hao ;
Wang, Huijin ;
Bai, Jieyun ;
Lu, Yaosheng ;
Long, Shun .
ANNALS OF TRANSLATIONAL MEDICINE, 2022, 10 (13)
[17]   Gradient Boosted Neural Decision Forest [J].
Dong, Manqing ;
Yao, Lina ;
Wang, Xianzhi ;
Benatallah, Boualem ;
Zhang, Shuai ;
Sheng, Quan Z. .
IEEE TRANSACTIONS ON SERVICES COMPUTING, 2023, 16 (01) :330-342
[18]   Predicting decompression surgery by applying multimodal deep learning to patients' structured and unstructured health data [J].
Jujjavarapu, Chethan ;
Suri, Pradeep ;
Pejaver, Vikas ;
Friedly, Janna ;
Gold, Laura S. ;
Meier, Eric ;
Cohen, Trevor ;
Mooney, Sean D. ;
Heagerty, Patrick J. ;
Jarvik, Jeffrey G. .
BMC MEDICAL INFORMATICS AND DECISION MAKING, 2023, 23 (01)
[19]   Predicting decompression surgery by applying multimodal deep learning to patients’ structured and unstructured health data [J].
Chethan Jujjavarapu ;
Pradeep Suri ;
Vikas Pejaver ;
Janna Friedly ;
Laura S. Gold ;
Eric Meier ;
Trevor Cohen ;
Sean D. Mooney ;
Patrick J. Heagerty ;
Jeffrey G. Jarvik .
BMC Medical Informatics and Decision Making, 23
[20]   Privet: A Privacy-Preserving Vertical Federated Learning Service for Gradient Boosted Decision Tables [J].
Zheng Y. ;
Xu S. ;
Wang S. ;
Gao Y. ;
Hua Z. .
IEEE Transactions on Services Computing, 2023, 16 (05) :3604-3620