Multi-task joint learning network based on adaptive patch pruning for Alzheimer's disease diagnosis and clinical score prediction

As a hot topic in brain diseases, diagnostic and clinical score prediction of subjects based on multi -modal images helps assess pathological stages and estimate disease progression. Since brain atrophy occurs only in localized regions, previous patch -based deep learning methods require pre -determination of discriminative locations in the brain. In other words, the features extracted from the pre -determined potential atrophy locations are not fully adapted to the tasks in subsequent stages. Besides, most methods focus only on singlemodal information with a single task, thus ignoring the intrinsic correlation between multi -modal information and multi -task variables. Furthermore, simply discarding subjects with incomplete clinical scores limits the number of available subjects. In this paper, we propose a multi -task joint learning network (MTJLN) for both brain disease diagnosis and clinical score prediction using multi -modal data and incomplete clinical scores. Specifically, we divided the brain images into 216 local patches covering all potential lesion locations. Then, an image patch pruning algorithm is designed for pruning the information -poor patches. The fine-grained multimodal image features based on patches and coarse -grained non -image features are fused in the middle layer and used to predict multi -task variables. The ingenious design of the weighted loss function enables subjects with incomplete clinical scores to participate in network training. The experimental results of 842 subjects from the ADNI database demonstrate that the proposed method can effectively predict the pathological stage and clinical score of subjects.