Bimodal learning via trilogy of skip-connection deep networks for diabetic retinopathy risk progression identification

被引:41
作者
Hua, Cam-Hao [1 ]
Thien Huynh-The [2 ]
Kim, Kiyoung [3 ]
Yu, Seung-Young [3 ]
Thuong Le-Tien [4 ]
Park, Gwang Hoon [1 ]
Bang, Jaehun [1 ]
Khan, Wajahat Ali [1 ]
Bae, Sung-Ho [1 ]
Lee, Sungyoung [1 ]
机构
[1] Kyung Hee Univ, Dept Comp Sci & Engn, Gyeonggi Do 17104, South Korea
[2] Kumoh Natl Inst Technol, ICT Convergence Res Ctr, Gumi, South Korea
[3] Kyung Hee Univ, Med Ctr, Dept Ophthalmol, Seoul 02447, South Korea
[4] Ho Chi Minh City Univ Technol, Dept Elect & Elect Engn, Ho Chi Minh City 700000, Vietnam
关键词
Bimodal learning; Diabetic Retinopathy risk progression; EMR-based attributes; Fundus photography; Retinal fundus image; Trilogy of skip-connection deep networks; DECISION-SUPPORT-SYSTEM; NEURAL-NETWORKS;
D O I
10.1016/j.ijmedinf.2019.07.005
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Background: Diabetic Retinopathy (DR) is considered a pathology of retinal vascular complications, which stays in the top causes of vision impairment and blindness. Therefore, precisely inspecting its progression enables the ophthalmologists to set up appropriate next-visit schedule and cost-effective treatment plans. In the literature, existing work only makes use of numerical attributes in Electronic Medical Records (EMR) for acquiring such kind of DR-oriented knowledge through conventional machine learning techniques, which require an exhaustive job of engineering most impactful risk factors. Objective: In this paper, an approach of deep bimodal learning is introduced to leverage the performance of DR risk progression identification. Methods: In particular, we further involve valuable clinical information of fundus photography in addition to the aforementioned systemic attributes. Accordingly, a Trilogy of Skip-connection Deep Networks, namely Tri-SDN, is proposed to exhaustively exploit underlying relationships between the baseline and follow-up information of the fundus images and EMR-based attributes. Besides that, we adopt Skip-Connection Blocks as basis components of the Tri-SDN for making the end-to-end flow of signals more efficient during feedforward and backpropagation processes. Results: Through a 10-fold cross validation strategy on a private dataset of 96 diabetic mellitus patients, the proposed method attains superior performance over the conventional EMR-modality learning approach in terms of Accuracy (90.6%), Sensitivity (96.5%), Precision (88.7%), Specificity (82.1%), and Area Under Receiver Operating Characteristics (88.8%). Conclusions: The experimental results show that the proposed Tri-SDN can combine features of different modalities (i.e., fundus images and EMR-based numerical risk factors) smoothly and effectively during training and testing processes, respectively. As a consequence, with impressive performance of DR risk progression recognition, the proposed approach is able to help the ophthalmologists properly decide follow-up schedule and subsequent treatment plans.
引用
收藏
页数:12
相关论文
共 48 条
[1]   A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification [J].
Al-antari, Mugahed A. ;
Al-masni, Mohammed A. ;
Choi, Mun-Taek ;
Han, Seung-Moo ;
Kim, Tae-Seong .
INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS, 2018, 117 :44-54
[2]  
[Anonymous], 1991, Ophthalmology, V98, P786
[3]  
[Anonymous], ABS181101238 CORR
[4]  
[Anonymous], 2018 INT JOINT C NEU
[5]  
[Anonymous], SCI REPORTS
[6]  
[Anonymous], ABS180709232 CORR
[7]  
[Anonymous], 2017, PLOS ONE
[8]  
[Anonymous], ADV NEURAL INFORM PR
[9]  
[Anonymous], ABS14090575 CORR
[10]  
[Anonymous], 2015, 3 INT C LEARN REPR I