Brain and lesion segmentation in multiple sclerosis using fully convolutional neural networks: A large-scale study

Objective: To investigate the performance of deep learning (DL) based on fully convolutional neural network (FCNN) in segmenting brain tissues in a large cohort of multiple sclerosis (MS) patients. Methods: We developed a FCNN model to segment brain tissues, including T2-hyperintense MS lesions. The training, validation, and testing of FCNN were based on ~1000 magnetic resonance imaging (MRI) datasets acquired on relapsing-remitting MS patients, as a part of a phase 3 randomized clinical trial. Multimodal MRI data (dual-echo, FLAIR, and T1-weighted images) served as input to the network. Expert validated segmentation was used as the target for training the FCNN. We cross-validated our results using the leave-one-center-out approach. Results: We observed a high average (95% confidence limits) Dice similarity coefficient for all the segmented tissues: 0.95 (0.92-0.98) for white matter, 0.96 (0.93-0.98) for gray matter, 0.99 (0.98-0.99) for cerebrospinal fluid, and 0.82 (0.63-1.0) for T2 lesions. High correlations between the DL segmented tissue volumes and ground truth were observed (R-2 > 0.92 for all tissues). The cross validation showed consistent results across the centers for all tissues. Conclusion: The results from this large-scale study suggest that deep FCNN can automatically segment MS brain tissues, including lesions, with high accuracy.