Abstract
BACKGROUND: Designing intensity-modulated radiotherapy (IMRT) plans for rectal cancer is complex and time-consuming. We used a three-dimensional (3D) multitask training U-Net (3D MT-U-Net) deep learning (DL) model to accurately predict radiotherapy dose distributions for rectal cancer. We aimed to achieve fully automated IMRT plans with improved efficiency and quality. METHODS: We developed a 3D MT-U-Net model that precisely captured dose distribution characteristics through pretraining and a multitask learning mechanism. In the multitask learning module, we additionally introduced the learning of gradient maps and isodose line maps to enhance the network's ability to extract semantic information from dose distributions. The patients were divided into a training set (n=99), an independent test set (n=26), and an external test set (n=15). The high-precision dose predictions were translated into automated optimization objectives by integrating clinical constraints to establish two fully automated optimization methods based on 3D voxel dose and dose-volume histogram (DVH) parameters. The Monte Carlo algorithm was used to perform dose calculations and achieve fully automated plans. Using manually designed plans as a reference, the dose distributions predicted by the model and generated by the automated plans were evaluated using the mean absolute error (MAE) and DVH parameters. RESULTS: Compared with the state-of-the-art DL architectures [pix2pix, DoseDiff (distance-aware diffusion model), and MD-Dose (diffusion model based on the Mamba)], the 3D MT-U-Net model demonstrated substantially improved prediction accuracy respectively, with notable reductions in MAE values for planning target volume (PTV)1 (0.016±0.023 vs. 0.038±0.024, 0.033±0.018, and 0.017±0.039) and PTV2 (0.022±0.007 vs. 0.033±0.019, 0.038±0.008, and 0.029±0.005). Both automated planning methods [voxel dose-based automated plan (AP_VD) and DVH-based automated plan (AP_DVH)] effectively protected organs at risk and maintained target coverage comparable to manual plans (MPs), with the voxel dose-based approach (AP_VD) demonstrating superior dosimetric performance-particularly significant reductions in small bowel V35 Gy, colon V45 Gy, bladder Dmean, and femoral head V30 Gy (P<0.05), achieving the highest average plan scores (81.24±6.15). Ablation studies confirmed that the multi-task learning mechanism incorporating both isodose line maps and gradient maps was key to enhancing model performance, with this combined configuration yielding the lowest MAE values. CONCLUSIONS: This study developed a fully automated IMRT plan design method for rectal cancer. This approach significantly improved the efficiency of designing IMRT plans and plan quality.