Abstract
This mixed-methods study evaluated clinicians' user experience (UX) with Generative AI (GenAI) in Electronic Health Record (EHR) systems across three clinical documentation tasks (Information Extraction, Summarization, and Speech-to-Text) at varying levels of user supervision (low, medium, high), focusing on workflow improvements, safety, and acceptable automation levels. Using conceptual prototyping in a usability study framework, we evaluated how incorporating GenAI into EHR could support the three documentation tasks at varying automation levels. A total of 38 clinicians interacted with the prototype and completed a questionnaire on task relevance, perceived importance, desired automation level, and EHR satisfaction. Both quantitative (descriptive statistics, Kruskal-Wallis tests, Spearman correlations) and qualitative (thematic) analyses were conducted with equal priority to explore preferences, perceived safety, and practical requirements. Clinicians showed positive reception to GenAI integration, particularly for streamlining documentation. While task relevance and importance were strongly correlated, EHR satisfaction did not significantly predict automation acceptance. Medium automation emerged as the preferred level, considered "safe with caution". Five key themes emerged from qualitative analysis: efficiency and quality benefits; system reliability concerns; safety and medico-legal considerations; automation bias and loss of nuance; and deployment requirements including adjustable settings and oversight. While clinicians welcome GenAI-driven documentation, they prefer moderate automation to balance efficiency with clinical control. Successful integration requires addressing safety concerns, conducting real-world trials, and mitigating potential biases and medico-legal challenges. These findings suggest a cautious but optimistic path forward for AI integration in EHR systems, emphasizing the importance of maintaining clinician oversight while leveraging automation benefits.