Alzheimer’s Disease (AD) is the most common form of dementia and often
difficult to diagnose due to the multifactorial etiology of dementia.
Recent works on neuroimaging-based computer-aided diagnosis with deep
neural networks (DNNs) showed that fusing structural magnetic
resonance images (sMRI) and fluorodeoxyglucose positron emission
tomography (FDG-PET) leads to improved accuracy in a study population
of healthy controls and subjects with AD. However, this result
conflicts with the established clinical knowledge that FDG-PET better
captures AD-specific pathologies than sMRI. Therefore, we propose a
framework for the systematic evaluation of multi-modal DNNs and
critically re-evaluate single- and multi-modal DNNs based on FDG-PET
and sMRI for binary healthy vs. AD, and three-way healthy/mild
cognitive impairment/AD classification. Our experiments demonstrate
that a single-modality network using FDG-PET performs better than MRI
(accuracy 0.91 vs 0.87) and does not show improvement when combined.
This conforms with the established clinical knowledge on AD
biomarkers, but raises questions about the true benefit of multi-modal
DNNs. We argue that future work on multi-modal fusion should
systematically assess the contribution of individual modalities
following our proposed evaluation framework. Finally, we encourage the
community to go beyond healthy vs. AD classification and focus on
differential diagnosis of dementia, where fusing multi-modal image
information conforms with a clinical need.
Medical Image Computing and Computer-Assisted Intervention