Abstract
Breast cancer survival prediction plays a significant role in better understanding the relationship between clinical features and demographic features to improve decision-making among clinicians. This is done with the help of machine learning models, which assist clinicians in making decisions for the improvement of patients. However, there has been an over-saturation of machine learning models in breast cancer survival prediction, of which the only distinction is the difference in data sampling used and better tuning parameters. This study explores the possibilities of a recent deep learning model called Transformers as a breast cancer survival prediction model. The breast cancer dataset used for the study ranges from 2006 until 2015. The dataset was retrieved from the University Malaya Medical Centre (UMMC), Kuala Lumpur, Malaysia. The dataset contains 26 variables comprising 24 categorical variables and two numerical variables. A comparison of performance matrices on Transformer models such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT Pretraining Approach) and ALBERT (A Lite BERT for Self-supervised Learning of Language Representations) to existing machine learning models such as random forest (RF), decision tree (DT), support vector machine (SVM), logistic regression (LR), k-nearest neighbours (KNN), artificial neural network (ANN) and extreme gradient boosting (XGB) was conducted. In terms of overall performance, the Transformer models produced scores that were close to those of other machine learning models. The best accuracy among the Transformer models was the ALBERT model at (81.1%) while the best accuracy among the machine learning models was the LR model at (83.4%). Based on the Transformer models’ close outcomes compared to machine learning models, Transformers can be introduced as an alternative prediction model in breast cancer survival studies. Clinical Trial Number: Not Applicable SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-025-03306-y.