Abstract
BACKGROUND: Natural Language Processing (NLP) models show promise in enhancing interpretation and triage of outpatient referrals across diverse specialties. OBJECTIVE: To conduct a systematic literature review and narrative synthesis of recent studies that utilized NLP-based models for triage-related tasks such as urgency prioritization, referral classification, and justification review. METHODS: Medline, Embase, Web of Science, and CINAHL databases were searched for articles published up to February 17 2024, limiting searches to the last 5 years prior to the search. All citations were imported into Covidence for duplicate removal and screening. We included studies that utilized NLP techniques to triage outpatient referrals to a specialist (medical or surgical), and included comparison to human triage. Abstracts and full texts were each screened independently by two reviewers. Data from each study were extracted independently by two reviewers using a standardized extraction form, including fields such as study design, dataset size, specialty, models tested, and outcomes reported. Results were synthesized narratively, organized by key themes focused on data, model and clinical applicability. Quality and risk of bias assessment was performed using the PROBAST-AI and Technology Readiness scales. RESULTS: A total of 4,225 titles and abstracts were reviewed resulting in 26 full-text reviews. A total of 10 studies were used for data extraction and synthesis. These studies spanned a wide range of medical specialties including surgery, medical specialties, and radiology. Tasks included predicting condition and priority level. Most domains were assessed as low or uncertain risk of bias. Outcome measures varied across studies, but overall, 7 studies reported high levels of accuracy compared to manual workflows. We summarized key differences in dataset preprocessing and augmentation, triage model, and feasibility and clinical applicability. CONCLUSION: NLP shows promise in augmenting human triage of outpatient referrals to specialty care. To realize the full potential of NLP for triage, future work should prioritize standardized reporting and prospective validation to support safe and effective integration into healthcare systems.