Comparing methods of detecting and segmenting unruptured intracranial aneurysms on TOF-MRAS: The ADAM challenge

被引:44
作者
Timmins, Kimberley M. [1 ]
van der Schaaf, Irene C. [2 ]
Bennink, Edwin [1 ]
Ruigrok, Ynte M. [3 ]
An, Xingle [4 ]
Baumgartner, Michael [5 ]
Bourdon, Pascal [6 ,7 ]
De Feo, Riccardo [8 ,9 ]
Di Noto, Tommaso [10 ,11 ]
Dubost, Florian [12 ]
Fava-Sanches, Augusto [13 ]
Feng, Xue [14 ]
Giroud, Corentin [12 ]
Hu, Minghui [16 ]
Jaeger, Paul F. [5 ]
Kaiponen, Juhana [9 ]
Klimont, Micha [15 ,17 ]
Li, Yuexiang [18 ]
Li, Hongwei [19 ,20 ]
Lin, Yi [18 ]
Loehr, Timo [19 ]
Ma, Jun [21 ]
Maier-Hein, Klaus H. [5 ,22 ]
Marie, Guillaume [10 ,11 ]
Menze, Bjoern [19 ,20 ]
Richiardi, Jonas [10 ,11 ]
Rjiba, Saifeddine [6 ,23 ]
Shah, Dhaval [24 ]
Shit, Suprosanna [24 ]
Tohka, Jussi [9 ]
Urruty, Thierry [6 ,7 ]
Walinska, Urszula [15 ]
Yang, Xiaoping [21 ]
Yang, Yunqiao [25 ]
Yin, Yin [16 ]
Velthuis, Birgitta K. [2 ]
Kuijf, Hugo J. [1 ]
机构
[1] Univ Med Ctr Utrecht, Image Sci Inst, Utrecht, Netherlands
[2] Univ Med Ctr Utrecht, Dept Radiol, Utrecht, Netherlands
[3] Univ Med Ctr Utrecht, UMC Utrecht Brain Ctr, Dept Neurol & Neurosurg, Utrecht, Netherlands
[4] China Elect Cloud Brain Tianjin Technol CO LTD, Tianjin 300309, Peoples R China
[5] German Canc Res Ctr, Div Med Image Comp, Heidelberg, Germany
[6] Univ Poitiers, Xlim Lab, CNRS, UMR 7252, Poitiers, France
[7] Univ Poitiers Hosp, CHU, Poitiers, France
[8] Sapienza Univ Roma, I-00184 Rome, Italy
[9] Univ Eastern Finland, AI Virtanen Inst Mol Sci, Kuopio 70210, Finland
[10] Lausanne Univ Hosp, Dept Radiol, Lausanne, Switzerland
[11] Univ Lausanne, Rue Du Bugnon 46, CH-1011 Lausanne, Switzerland
[12] Zelos Mediacorp, Rotterdam, Netherlands
[13] Univ Hosp LMU, Inst Neuroradiol, Munich, Germany
[14] Univ Virginia, Biomed Engn, Thornton Hall,POB 400259, Charlottesville, VA 22904 USA
[15] Inteneural Networks, Warsaw, Poland
[16] Union Strong Beijing Technol Co Ltd, Beijing Econ Technol Dev Area, DaZu Plaza T3-901,2 Ronghua South Rd, Beijing 100176, Peoples R China
[17] Poznan Univ Med Sci, Dept Radiol, Poznan, Poland
[18] Tencent Jarvis Lab, Shenzhen, Guangdong, Peoples R China
[19] Tech Univ Munich, Dept Comp Sci, Munich, Germany
[20] Univ Zurich, Dept Quantitat Biomed, Zurich, Switzerland
[21] Nanjing Univ Sci & Technol, Dept Math, Nanjing 210094, Jiangsu, Peoples R China
[22] Heidelberg Univ Hosp, Dept Radiat Oncol, Pattern Anal & Learning Grp, Heidelberg, Germany
[23] Canon Med Syst, Puteaux La Defense, France
[24] Tech Univ Munich, Dept Informat, Munich, Germany
[25] Huazhong Univ Sci & Technol, Wuhan, Hubei, Peoples R China
基金
中国国家自然科学基金; 欧盟地平线“2020”; 欧洲研究理事会; 芬兰科学院;
关键词
Challenge; Segmentation; Detection; Aneurysms; Angiography; MAGNETIC-RESONANCE ANGIOGRAPHY; INTEROBSERVER VARIABILITY; SUBARACHNOID HEMORRHAGE; CT ANGIOGRAPHY; PHASES SCORE; RISK; PREDICTION; MANAGEMENT; HISTORY; RUPTURE;
D O I
10.1016/j.neuroimage.2021.118216
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Accurate detection and quantification of unruptured intracranial aneurysms (UIAs) is important for rupture risk assessment and to allow an informed treatment decision to be made. Currently, 2D manual measures used to assess UIAs on Time-of-Flight magnetic resonance angiographies (TOF-MRAs) lack 3D information and there is substantial inter-observer variability for both aneurysm detection and assessment of aneurysm size and growth. 3D measures could be helpful to improve aneurysm detection and quantification but are time-consuming and would therefore benefit from a reliable automatic UIA detection and segmentation method. The Aneurysm Detection and segMentation (ADAM) challenge was organised in which methods for automatic UIA detection and segmentation were developed and submitted to be evaluated on a diverse clinical TOF-MRA dataset. A training set (113 cases with a total of 129 UIAs) was released, each case including a TOF-MRA, a structural MR image (T1, T2 or FLAIR), annotation of any present UIA(s) and the centre voxel of the UIA(s). A test set of 141 cases (with 153 UIAs) was used for evaluation. Two tasks were proposed: (1) detection and (2) segmentation of UIAs on TOF-MRAs. Teams developed and submitted containerised methods to be evaluated on the test set. Task 1 was evaluated using metrics of sensitivity and false positive count. Task 2 was evaluated using dice similarity coefficient, modified hausdorff distance (95th percentile) and volumetric similarity. For each task, a ranking was made based on the average of the metrics. In total, eleven teams participated in task 1 and nine of those teams participated in task 2. Task 1 was won by a method specifically designed for the detection task (i.e. not participating in task 2). Based on segmentation metrics, the top two methods for task 2 performed statistically significantly better than all other methods. The detection performance of the top-ranking methods was comparable to visual inspection for larger aneurysms. Segmentation performance of the top ranking method, after selection of true UIAs, was similar to interobserver performance. The ADAM challenge remains open for future submissions and improved submissions, with a live leaderboard to provide benchmarking for method developments at https://adam.isi.uu.nl/ .
引用
收藏
页数:21
相关论文
共 82 条
[1]  
[Anonymous], 2013, ICML
[2]   Computerized detection of intracranial aneurysms for three-dimensional MR angiography: Feature extraction of small protrusions based on a shape-based difference image technique [J].
Arimura, H ;
Li, G ;
Korogi, Y ;
Hirai, T ;
Katsuragawa, S ;
Yamashita, Y ;
Tsuchiya, K ;
Doi, K .
MEDICAL PHYSICS, 2006, 33 (02) :394-401
[3]   ELAPSS score for prediction of risk of growth of unruptured intracranial aneurysms [J].
Backes, Daan ;
Rinkel, Gabriel J. E. ;
Greving, Jacoba P. ;
Velthuis, Birgitta K. ;
Murayama, Yuichi ;
Takao, Hiroyuki ;
Ishibashi, Toshihiro ;
Igase, Michiya ;
terbrugge, Karel G. ;
Agid, Ronit ;
Jaeaeskelaeinen, Juha E. ;
Lindgren, Antti E. ;
Koivisto, Timo ;
zu Fraunberg, Mikael von und ;
Matsubara, Shunji ;
Moroi, Junta ;
Wong, George K. C. ;
Abrigo, Jill M. ;
Igase, Keiji ;
Matsumoto, Katsumi ;
Wermer, Marieke J. H. ;
van Walderveen, Marianne A. A. ;
Algra, Ale ;
Vergouwen, Mervyn D. I. .
NEUROLOGY, 2017, 88 (17) :1600-1606
[4]   PHASES Score for Prediction of Intracranial Aneurysm Growth [J].
Backes, Daan ;
Vergouwen, Mervyn D. I. ;
Groenestege, Andreas T. Tiel ;
Bor, A. Stijntje E. ;
Velthuis, Birgitta K. ;
Greving, Jacoba P. ;
Algra, Ale ;
Wermer, Marieke J. H. ;
van Walderveen, Marianne A. A. ;
terBrugge, Karel G. ;
Agid, Ronit ;
Rinkel, Gabriel J. E. .
STROKE, 2015, 46 (05) :1221-1226
[5]  
Baumgartner M., 2020, RETINA U NET ANEURYS
[6]  
Berrada L., 2018, SMOOTH LOSS FUNCTION
[7]   Modality agnostic intracranial aneurysm detection through supervised vascular surface classification [J].
Bizjak, Ziga ;
Likar, Bostjan ;
Pernus, Franjo ;
Spiclin, Ziga .
MEDICAL IMAGING 2021: COMPUTER-AIDED DIAGNOSIS, 2021, 11597
[8]   Optimal screening strategy for familial intracranial aneurysms A cost-effectiveness analysis [J].
Bor, A. Stijntje E. ;
Koffijberg, Hendrik ;
Wermer, Marieke J. H. ;
Rinkel, Gabriel J. E. .
NEUROLOGY, 2010, 74 (21) :1671-1679
[9]   Long-term, serial screening for intracranial aneurysms in individuals with a family history of aneurysmal subarachnoid haemorrhage: a cohort study [J].
Bor, A. Styntje E. ;
Rinke, Gabriel J. E. ;
van Norden, Jeroen ;
Wermer, Marieke J. H. .
LANCET NEUROLOGY, 2014, 13 (04) :385-392
[10]   Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening [J].
Brown, Robert D., Jr. ;
Broderick, Jrioseph P. .
LANCET NEUROLOGY, 2014, 13 (04) :393-404