Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914785427193856 |
|---|---|
| author | Rony, Md Main Uddin Haque, Md Mahfuzul Ali, Mohammad Alam, Ahmed Shatil Hassan, Naeemul |
| author_facet | Rony, Md Main Uddin Haque, Md Mahfuzul Ali, Mohammad Alam, Ahmed Shatil Hassan, Naeemul |
| contents | In the digital age, the prevalence of misleading news headlines poses a significant challenge to information integrity, necessitating robust detection mechanisms. This study explores the efficacy of Large Language Models (LLMs) in identifying misleading versus non-misleading news headlines. Utilizing a dataset of 60 articles, sourced from both reputable and questionable outlets across health, science & tech, and business domains, we employ three LLMs- ChatGPT-3.5, ChatGPT-4, and Gemini-for classification. Our analysis reveals significant variance in model performance, with ChatGPT-4 demonstrating superior accuracy, especially in cases with unanimous annotator agreement on misleading headlines. The study emphasizes the importance of human-centered evaluation in developing LLMs that can navigate the complexities of misinformation detection, aligning technical proficiency with nuanced human judgment. Our findings contribute to the discourse on AI ethics, emphasizing the need for models that are not only technically advanced but also ethically aligned and sensitive to the subtleties of human interpretation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_03153 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines Rony, Md Main Uddin Haque, Md Mahfuzul Ali, Mohammad Alam, Ahmed Shatil Hassan, Naeemul Computation and Language Computers and Society Machine Learning In the digital age, the prevalence of misleading news headlines poses a significant challenge to information integrity, necessitating robust detection mechanisms. This study explores the efficacy of Large Language Models (LLMs) in identifying misleading versus non-misleading news headlines. Utilizing a dataset of 60 articles, sourced from both reputable and questionable outlets across health, science & tech, and business domains, we employ three LLMs- ChatGPT-3.5, ChatGPT-4, and Gemini-for classification. Our analysis reveals significant variance in model performance, with ChatGPT-4 demonstrating superior accuracy, especially in cases with unanimous annotator agreement on misleading headlines. The study emphasizes the importance of human-centered evaluation in developing LLMs that can navigate the complexities of misinformation detection, aligning technical proficiency with nuanced human judgment. Our findings contribute to the discourse on AI ethics, emphasizing the need for models that are not only technically advanced but also ethically aligned and sensitive to the subtleties of human interpretation. |
| title | Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines |
| topic | Computation and Language Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2405.03153 |