Multi-View Multi-Modal Feature Embedding for Endomicroscopy Mosaic Classification

被引:9
作者
Gu, Yun [1 ]
Yang, Jie [1 ]
Yang, Guang-Zhong [2 ]
机构
[1] Shanghai Jiao Tong Univ, Sch Biomed Engn, Shanghai 200030, Peoples R China
[2] Imperial Coll London, Hamlyn Ctr Robot Surg, London, England
来源
PROCEEDINGS OF 29TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, (CVPRW 2016) | 2016年
基金
英国工程与自然科学研究理事会;
关键词
RETRIEVAL;
D O I
10.1109/CVPRW.2016.166
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Probe-based confocal laser endomicroscopy (pCLE) is an emerging tool for epithelial cancer diagnosis, which enables in vivo microscopic imaging during endoscopic procedures. As a new technique, definite clinical diagnosis is still referenced to the gold standard histology images. In this paper, we propose a Multi-View Multi-Modal Embedding framework (MVMME) to learn representative features for pCLE videos exploiting both pCLE mosaic and histology images. Each pCLE mosaic is represented by multiple feature representations including SIFT, Texton and HoG. A latent space is discovered by embedding the visual features from both mosaics and histology images in a supervised scheme. The features extracted from the latent spaces can make use of multi-modal imaging sources that are more discriminative than unimodal features from mosaics alone. The experiments based on real pCLE datasets demonstrate that our approach outperforms, with statistical significance, several single-view or single-modal methods. A binary classification accuracy of 96% has been achieved.
引用
收藏
页码:1315 / 1323
页数:9
相关论文
共 23 条
[1]   ENDOMICROSCOPIC VIDEO RETRIEVAL USING MOSAICING AND VISUAL WORDS [J].
Andre, B. ;
Vercauteren, T. ;
Buchner, A. M. ;
Wallace, M. B. ;
Ayache, N. .
2010 7TH IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING: FROM NANO TO MACRO, 2010, :1419-1422
[2]   ENDOMICROSCOPIC IMAGE RETRIEVAL AND CLASSIFICATION USING INVARIANT VISUAL FEATURES [J].
Andre, B. ;
Vercauteren, T. ;
Perchant, A. ;
Buchner, A. M. ;
Wallace, M. B. ;
Ayache, N. .
2009 IEEE INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING: FROM NANO TO MACRO, VOLS 1 AND 2, 2009, :346-+
[3]   Learning Semantic and Visual Similarity for Endomicroscopy Video Retrieval [J].
Andre, Barbara ;
Vercauteren, Tom ;
Buchner, Anna M. ;
Wallace, Michael B. ;
Ayache, Nicholas .
IEEE TRANSACTIONS ON MEDICAL IMAGING, 2012, 31 (06) :1276-1288
[4]  
André B, 2011, LECT NOTES COMPUT SC, V6893, P297, DOI 10.1007/978-3-642-23626-6_37
[5]   A smart atlas for endomicroscopy using automated video retrieval [J].
Andre, Barbara ;
Vercauteren, Tom ;
Buchner, Anna M. ;
Wallace, Michael B. ;
Ayache, Nicholas .
MEDICAL IMAGE ANALYSIS, 2011, 15 (04) :460-476
[6]  
[Anonymous], P INT C COMP VIS
[7]  
[Anonymous], 2011, ACM T INTEL SYST TEC, DOI DOI 10.1145/1961189.1961199
[8]   Imaging breast cancer morphology using probe-based confocal laser endomicroscopy: towards a real-time intraoperative imaging tool for cavity scanning [J].
Chang, Tou Pin ;
Leff, Daniel R. ;
Shousha, Sami ;
Hadjiminas, Dimitri J. ;
Ramakrishnan, Rathi ;
Hughes, Michael R. ;
Yang, Guang-Zhong ;
Darzi, Ara .
BREAST CANCER RESEARCH AND TREATMENT, 2015, 153 (02) :299-310
[9]   Histograms of oriented gradients for human detection [J].
Dalal, N ;
Triggs, B .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2005, :886-893
[10]   Image Annotation by Latent Community Detection and Multikernel Learning [J].
Gu, Yun ;
Qian, Xueming ;
Li, Qing ;
Wang, Meng ;
Hong, Richang ;
Tian, Qi .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2015, 24 (11) :3450-3463