Abstract
BACKGROUND: Prognosis of future risk of dementia from neuroimaging and cognitive data is important for optimizing clinical management for patients at early stage of Alzheimer's disease (AD). However, existing studies lack an efficient way to integrate longitudinal information from both modalities to improve prognosis performance. OBJECTIVE: In this study, we aim to develop and evaluate an explainable deep learning-based framework to predict mild cognitive impairment (MCI) to AD conversion within four years using longitudinal whole-brain 3D MRI and neurocognitive tests. METHODS: We proposed a two-stage framework that first uses a 3D convolutional neural network to extract single-timepoint MRI-based AD-related latent features, followed by multi-modal longitudinal feature concatenation and a 1D convolutional neural network to predict the risk of future dementia onset in four years. RESULTS: The proposed deep learning framework showed promising to predict MCI to AD conversion within 4 years using longitudinal whole-brain 3D MRI and cognitive data without extracting regional brain volumes or cortical thickness, reaching a balanced accuracy of 0.834, significantly improved from models trained from single timepoint or single modality. The post hoc model explainability revealed heatmap indicating regions that are important for predicting future risk of AD. CONCLUSIONS: The proposed framework sets the stage for future studies for using multi-modal longitudinal data to achieve optimal prediction for prognosis of AD onset, leading to better management of the diseases, thereby improving the quality of life.