Abstract
BACKGROUND: Artificial intelligence (AI) chatbots perform well in answering English cancer questions. For Spanish, their performance is unknown and may differ by free vs. paywall versions. METHODS: We evaluated the quality (range: 1-5 points), actionability (range: 0-100%), and readability (range: 1-13 grades) of six popular AI chatbots in responding to the 15 most searched Spanish questions regarding breast, prostate, and colon cancer. RESULTS: The quality of overall AI chatbot responses was good (mean [95% CI]: 3.5 [3.4-3.6] points), while the actionability was low (mean [95% CI]: 35.6% [30.8%-40.3%]). The readability was high-school-level (mean [95% CI]: 9.2 [8.8-9.6] grades), not concordant with the American Medical Association recommendation (≤ 6th grade). The quality, actionability, and readability did not differ by free and paywall versions (p > 0.05). CONCLUSION: Our findings suggested AI chatbots may generate good-quality responses to Spanish cancer questions, regardless of free or paywall versions. However, further improvement in actionability and readability is needed to benefit Spanish-speaking patients.