Benchmarking Hook and Bait Urdu news dataset for domain-agnostic and multilingual fake news detection using large language models

使用大型语言模型对 Hook and Bait 乌尔都语新闻数据集进行基准测试,以评估其在领域无关和多语言虚假新闻检测方面的性能。

阅读:1

Abstract

Fake News (FN) prevalence on Online Social Networks (OSNs) and online websites is a worldwide issue. The previous studies on Fake News Detection (FND) have focused on rich-resource languages with limited relevance to users other than native speakers. Despite the progressing multilingual approaches, FND in low-resource languages remains obscure due to a lack of large-sized annotated corpora of real-world news. Large Language Models (LLMs) have emerged as a promising solution for multilingual FND. This study leverages the power of LLMs for an automated mechanism compared to traditional feature extraction methods. We curate the first large-sized multi-domain corpus, the Hook and Bait Urdu, with 78,409 fake and true news, and fine-tune the LLaMA 2 model for our proposed approach. We implement the curated dataset for two experiments. First, this study evaluates the dataset for unimodal text-based Urdu FND. The proposed LLaMA-based approach shows an accuracy of 0.978 and an F1-score of 0.971. For our second experiment, we fine-tuned the LLaMA 2-based framework for multilingual FND using the curated dataset (in Urdu) and the ISOT Fake News dataset (in English). Analytical and prediction performance comparisons with the previous studies validate the efficacy of the proposed framework with an accuracy of 0.984 and an F1-score of 0.980. The lightweight LoRA fine-tuning method, with  0.032% trainable parameters, ensured robust data handling, computational efficiency while leveraging early stopping and optimized hyperparameters for reliable and high-performing monolingual and multilingual FND. The real-world news dataset is publicly available for developing an automated FND mechanism to curb the threat of FN and related cybercrimes.

特别声明

1、本页面内容包含部分的内容是基于公开信息的合理引用;引用内容仅为补充信息,不代表本站立场。

2、若认为本页面引用内容涉及侵权,请及时与本站联系,我们将第一时间处理。

3、其他媒体/个人如需使用本页面原创内容,需注明“来源:[生知库]”并获得授权;使用引用内容的,需自行联系原作者获得许可。

4、投稿及合作请联系:info@biocloudy.com。