Doppler Image-Based Weakly-Supervised Vascular Ultrasound Segmentation with Transformer

Vascular segmentation in ultrasound (US) images faces labor-intensive labeling procedures and performance degradation due to unsatisfied image quality. Herein, we propose to use the Doppler image for vascular segmentation with an incremental Transformer structure. First, local features within the image patch are extracted by a convolutional neural network (CNN) patch embedding layer and further encoded by a multi-level Transformer to enhance the global dependencies from coarse to fine. A multi-level CNN decoder is introduced to decode corresponding features. Doppler imaging is capable of blood visualization and indicating the positional and structural information used as the pseudo label. A conditional random field (CRF) module and shape similarity loss function are introduced to improve the effectiveness of the Doppler images. The segmentation accuracy of the radial artery and carotid artery datasets can achieve 78.8% and 81.9% in Dice, with 63.5% and 53.7% accuracy in noisy labels. In addition, the framework can be generalized to unseen data without related Doppler images.