Abstract
BACKGROUND: As various machine learning (ML) algorithms have become more popular in orthopedic surgery, the research quality of these models requires further evaluation, and the methodological quality of the models still needs to be clarified. This study aimed to comprehensively analyze and evaluate the potential bias and applicability of existing studies of supervised ML-driven prognostic risk prediction models focusing on total knee arthroplasty/total hip arthroplasty individuals. METHODS: The China National Knowledge Infrastructure, Wanfang, PubMed, Web of Science, Cochrane Library, and Embase databases were searched from inception to January 20, 2024. The following data were extracted from the selected studies: author, year, joints, sample size, data source, study design, ML methods, ML performance, and primary outcome. The PROBAST checklist was applied to evaluate the risk of bias and applicability in prediction model studies. The protocol is registered in the PROSPERO database (CRD42024501747). RESULTS: A total of 2909 indexed records were obtained and 32 studies were included, 30 of which had models for internal validation only, 1 study for development and validation, and 1 study for external validation only. The PROBAST evaluation results showed that 1 externally validated model and 29/31 (93%) development models were rated as having a high risk of bias. There was a high risk of bias in the participant and analysis domains. CONCLUSION: Almost all supervised ML models have the potential for a high bias risk. Factors contributing to a high bias risk include inadequate sample size, missing data during recruitment, model overfitting, and limited external validation. Adhering to strict standards and implementing comprehensive improvements when constructing prognosis models using supervised ML is crucial.