Abstract
Artificial intelligence (AI)-based multi-modal fusion algorithms are pivotal in emulating clinical practice by integrating data from diverse sources. However, most of the existing multi-modal models focus on designing new modal fusion methods, ignoring critical role of feature representation. Enhancing feature representativeness can address the noise caused by modal heterogeneity at the source, enabling high performance even with small datasets and simple architectures. Here, we introduce DeepOmix-FLEX (Fusion with Learning Enhanced feature representation for X-modal or FLEX in short), a multi-modal fusion model that integrates clinical data, proteomic data, metabolomic data, and pathology images across different scales and modalities, with a focus on advanced feature learning and representation. FLEX contains a Feature Encoding Trainer structure that can train feature encoding, thus achieving fusion of inter-feature and inter-modal. FLEX achieves a mean AUC of 0.887 for prediction of chronic kidney disease progression on an internal dataset, exceeding the mean AUC of 0.727 using conventional clinical variables. Following external validation and interpretability analyses, our model demonstrated favorable generalizability and validity, as well as the ability to exploit markers. In summary, FLEX highlights the potential of AI algorithms to integrate multi-modal data and optimize the allocation of healthcare resources through accurate prediction.