Abstract
This study investigates how people perceive and evaluate true and false news in their natural environment using a novel experience sampling methodology with real-time news streaming. The sample consisted of 110 participants who evaluated news headlines on their smartphones throughout the day for two weeks, receiving notifications when new content was published. The study employed a custom-developed server that captured RSS feeds from major news outlets. The server used AI (the Open AI "gpt-4-0125-preview" model) to generate modified versions of news stories on the fly, including misinformation variants. Participants evaluated news live under experimentally manipulated conditions that included time constraints for reading the news. They also provided information about their environmental context and individual characteristics. The results showed that false news items were generally rated as less accurate than true news, but this discernment decreased under time constraint. Higher digital literacy and greater satisfaction with the political system were associated with rating false news as less accurate, whereas higher dogmatism was linked to higher perceived accuracy of false news. Familiarity was also related to higher accuracy ratings for both true and false news, meaning participants rated both types of news as more accurate when they felt familiar with it, consistent with the illusory truth effect. Integrating experimental AI-guided, real-time news generation and streaming offers a novel and much-needed approach to studying misinformation perception, providing externally valid insights into how real-world factors influence people's ability to detect and respond to false information in their daily lives.