Abstract
BACKGROUND & AIMS: Little is known about the ability of natural language processing (NLP) to extract meaningful information from free-text gastroenterology reports for secondary use. METHODS: We randomly selected 500 linked colonoscopy and pathology reports from 10,798 nonsurveillance colonoscopies to train and test the NLP system. By using annotation by gastroenterologists as the reference standard, we assessed the accuracy of an open-source NLP engine that processed and extracted clinically relevant concepts. The primary outcome was the highest level of pathology. Secondary outcomes were location of the most advanced lesion, largest size of an adenoma removed, and number of adenomas removed. RESULTS: The NLP system identified the highest level of pathology with 98% accuracy, compared with triplicate annotation by gastroenterologists (the standard). Accuracy values for location, size, and number were 97%, 96%, and 84%, respectively. CONCLUSIONS: The NLP can extract specific meaningful concepts with 98% accuracy. It might be developed as a method to further quantify specific quality metrics.