Automated Macular OCT Retinal Surface Segmentation in Cases of Severe Glaucoma Using Deep Learning

Glaucoma is one of the leading causes of permanent blindness due to optic nerve damage. Optical coherence tomography (OCT) has become an important clinical tool for assessing structural damage from the loss of neurons. Traditional 2D and 3D methods have been successfully applied to quantify inner retinal layer thickness. However, these methods show less reliable segmentation in severe glaucoma when the retinal layers have become thin and violate algorithm assumptions. Deep learning (DL) is an alternative image analysis approach due to its powerful ability to extract features directly from data. State-of-the-art DL segmentation approaches can achieve sub-pixel accuracy at multiple retinal surfaces in OCT scans from normal eyes. However, limitations, such as spike-like segmentation errors (showing as high Hausdorff distances) and lack of contextual information from the input image, still need to be improved. To address these limitations, three novel solutions were proposed in this study. First, for data augmentation, we reconstructed more B-scans by reassembling A-scans at the vertical and jittered planes to expose DL to a greater variety of features encountered in OCT. Second, smoothed and contrast-enhanced images of each three adjacent B-scans were concatenated to provide a six-channel input image stack to the neural network with contextual information. Finally, we merged the predicted surfaces from both horizontal and vertical B-scans while maintaining retinal topological order. In our independently tested dataset, which included eyes with severe glaucoma, the proposed approach outperformed the state-of-the-art methods in mean absolute surface distances, Dice coefficients, and Hausdorff distance at multiple surfaces.