Deep Low-Rank Multimodal Fusion with Inter-modal Distribution Difference Constraint for ASD Diagnosis

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by complex symptoms, which makes ASD difficult to be identified. Combining different brain imaging modalities to provide complementary information has been extensively used in the diagnosis of brain disorders. However, it is still very difficult to fully integrate different modalities by capturing the complex connections between different modalities. To solve this problem, we propose a deep low-rank multimodal fusion (DLMF) network that takes into account distribution discrepancy between different modalities. This network aims to learn the complex connections between rest-state functional magnetic resonance imaging (rs-fMRI) and structural magnetic resonance imaging (sMRI) in order to effectively perform multimodal identification of Autism Spectrum Disorder (ASD). Firstly, two different networks are used to extract the features that represent complex information in the rs-fMRI and sMRI data. Then, a measurement function is proposed to quantify distribution discrepancy between different modalities. This measurement function is then incorporated into the loss function of our low-rank multimodal fusion network. Therefore, our method can reduce the distribution discrepancy between different modalities through joint learning from rs-fMRI and sMRI data. The classifier in our approach adopts Support Vector Machines (SVM). The proposed network was trained with the new loss function using an end-to-end training approach. We verify the effectiveness of our method on a publicly available multimodal dataset: ABIDE database. Experimental results show that our methods are superior to several of the most advanced ASD diagnostic methods currently available.