Abstract
Synergistic drug combinations are often used to treat cancer. Experimental exploration of all possibilities is expensive. Deep learning (DL) offers a potential alternative for predicting drug pair synergy in specific cell lines. However, current methods often suffer from data leakage and lack systematic ablation studies. We propose SynVerse, a comprehensive evaluation framework featuring four data-splitting strategies to assess DL model generalizability and three ablation studies: module-based, feature shuffling, and a novel network-based approach to disentangle factors influencing performance. We evaluated sixteen models incorporating eight drug- and cell line-specific features, five preprocessing techniques, and two encoders. Our analysis revealed that no model outperformed a baseline using one-hot encoding. Biologically meaningful drug or cell line features and drug-drug interactions did not drive predictive performance. All models showed poor generalization to unseen drugs and cell lines. SynVerse highlights the need for substantial improvements before computational predictors can reliably support experimental and clinical settings.