Abstract
Background Artificial intelligence (AI) is increasingly being used in healthcare, particularly for interpreting complex medical queries. However, conventional AI models often generate inaccurate or irrelevant responses that are commonly termed hallucinations, which may compromise patient safety. To address this, our study introduces a modified retrieval-augmented generation (RAG) framework tailored for the urology domain to enhance contextual relevance and accuracy in AI-generated responses. Methodology We developed a context-aware RAG system integrating PubMedBERT embeddings for encoding and retrieving urological literature stored in a Pinecone vector database. The system uses named entity recognition for domain-specific query filtering and incorporates dynamic memory to retain contextual flow during interactions. Response generation is powered by the LLaMA3-8B model via LangChain. A custom dataset of urology-related queries was used for evaluation, with a large language model-based scoring using the Deepseek-R1 model. Results The proposed framework demonstrated a significant reduction in hallucinations, with responses being more contextually relevant and evidence-based. Compared to baseline models, our system achieved an 89% performance improvement in generating medically appropriate answers. Integration of memory modules and named entity filtering further improved precision and reliability. Conclusions Our RAG-enhanced system shows strong potential for clinical use by producing trustworthy, context-aware responses in urology. It addresses key challenges in medical AI, including hallucination mitigation and domain relevance. Future work will focus on reducing inference latency and improving automated validation without manual oversight.