Model-based segmentation using neural network-based boundary detectors: Application to prostate and heart segmentation in MR images

Model -based segmentation (MBS) is a variant of active surfaces and active shape models that has successfully been used to segment anatomical structures such as the heart or the brain. We propose to integrate neural networks (NNs) into MBS for boundary detection. We formulate boundary detection as a regression task and use a NN to predict the distances between a surface mesh and the corresponding boundary points. The proposed approach has been applied to two tasks - prostate segmentation in MR images and the segmentation of the left and right ventricle in MR images. For the first task, data from the Prostate MR Image Segmentation 2012 (PROMISE12) challenge has been used. For the second task, a diverse database with cardiac MR images from six clinical sites has been used. We compare the results to the popular U -net approaches using the nnU-net implementation that is among the top performing segmentation algorithms in various challenges. In cross validation experiments, the mean Dice scores are very similar and no statistically significant difference is observed. On the PROMISE12 test set, nnU-net Dice scores are significantly better. This is achieved by using an ensemble of 2D and 3D U -nets to generate the final segmentation, a concept that may also be adapted to NN -based boundary detection in the future. While the U -net provides a voxel labeling, our approach provides a 3D surface mesh with pre -defined mesh topology, establishes correspondences with respect to the reference mesh, avoids isolated falsely segmented regions and ensures proper connectivity of different regions.