Predicting stroke outcome: A case for multimodal deep learning methods with tabular and CT Perfusion data

Motivation: Acute ischemic stroke is one of the leading causes of morbidity and disability worldwide, often followed by a long rehabilitation period. To improve and personalize stroke rehabilitation, it is essential to provide a reliable prognosis to caregivers and patients. Deep learning techniques might improve the predictions by incorporating different data modalities. We present a multimodal approach to predict the functional status of acute ischemic stroke patients after their discharge based on tabular data and CT perfusion imaging.Methods: We conducted experiments on tabular, imaging, and multimodal deep learning architectures to predict dichotomized mRS scores 3 months after the event. The dataset was collected from a Dutch hospital and includes 98 CVA patients with a visible occlusion on their CT perfusion scan. Tabular data is based on the Dutch Acute Stroke Audit data, and imaging data consists of summed-up CT perfusion maps.Results: On the tabular data, TabNet outperformed our baselines with an AUC of 0.71, while ResNet-10 on the imaging data performed comparably with an AUC of 0.70. Our implementation of the multimodal DAFT architecture outperforms baselines as well as comparable studies by achieving an 0.75 AUC, and 0.80 F1 score. This was achieved with a final model of less than a hundred thousand optimizable parameters, and a dataset less than half the size of reference papers.Conclusion: Overall, we demonstrate the feasibility of predicting the functional outcome for ischemic stroke patients and the usability of multimodal deep learning architectures for this purpose.