A Novel original feature fusion network for joint diabetic retinopathy and diabetic Macular edema grading

Diabetic retinopathy (DR) and its complication diabetic macular edema (DME) are the leading cause of permanent blindness in the working-age population worldwide. Automated grading of DR and DME enables ophthalmologists to carry out tailored treatments to patients in early stages of their diseases. However, most of the current works only focus on the grading of a single disease, ignoring the relationship between DR and DME, and the traditional convolutional architectures face the problem that they can not capture long-distance dependencies despite of the effectiveness of extracting image features. To this end, we propose an original feature fusion network (OFFNet) for joint DR and DME grading based on the idea of key-value query, which consists of a specific feature extraction module (SFEM) based on self-attention and an original feature fusion module (OFFM) based on cross-attention. The proposed OFFNet enjoys several merits. First, to the best of our knowledge, this is the first joint grading effort based on the idea of key-value query. Second, OFFNet only needs image-level supervision, which can facilitate the acquisition of training data, rather than patch-level or pixel-level supervision. Third, OFFNet has obvious advantages in capturing long-distance dependencies. Extensive experiments on two public datasets Messidor and 2018 IDRiD challenge show that our method outperforms other joint grading methods on joint grading accuracy and the ability of capturing long-distance dependencies.