Abstract
In drug discovery, identifying molecules with desired pharmacological properties remains challenging, as conventional methods often rely on exhaustive trial-and-error and limited exploration of chemical space. Here, we present STELLA, a metaheuristics-based generative molecular design framework that combines an evolutionary algorithm for fragment-based chemical space exploration with a clustering-based conformational space annealing method for efficient multi-parameter optimization. Additionally, it leverages deep learning models for accurate prediction of pharmacological properties. Our case study, which focuses on docking score and quantitative estimate of drug-likeness as primary objectives, demonstrates that STELLA generates 217% more hit candidates with 161% more unique scaffolds and achieves more advanced Pareto fronts compared to REINVENT 4. In performance evaluations optimizing 16 properties simultaneously for MolFinder, REINVENT 4, and STELLA, STELLA consistently outperforms the control methods by achieving better average objective scores and exploring a broader region of chemical space. The results highlight STELLA's superior performance in both efficient exploration of chemical space and multi-parameter optimization, indicating that STELLA is a powerful tool for de novo molecular design.