Saved in:
| Main Authors: | , |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2026
|
| Online Access: | https://doi.org/10.5281/zenodo.18657875 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866902101532082176 |
|---|---|
| author | Picazo-Sanchez, Pablo Ortiz Martin, Lara |
| author_facet | Picazo-Sanchez, Pablo Ortiz Martin, Lara |
| contents | <p>This dataset supports the paper <strong>'</strong><strong>Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature</strong><strong>'</strong>. It contains a collection of bibliography entries generated by LLMs to study the frequency and nature of citation hallucinations.</p> <p> </p> <p>The dataset includes bibliographic entries generated by a diverse set of Large Language Models (LLMs), covering different architectures and scales. Specifically, the following models were evaluated:</p> <ul> <li> <p><strong>Microsoft Phi Family:</strong> <code>phi:2.7b:4q</code>, <code>phi3.5:3.8b:4q</code></p> </li> <li> <p><strong>Meta Llama Family:</strong> <code>llama2:7b:4q</code>, <code>llama3.3:70b:4q</code></p> </li> <li> <p><strong>Google Gemma Family:</strong> <code>gemma:7b:4q</code>, <code>gemma2:9b:4q</code></p> </li> <li> <p><strong>Specialized/Other Models:</strong> <code>dolphin-mistral:7b:4q</code>, <code>command-r:35b:4q</code>, <code>deepSeek-r1:8b</code></p> </li> </ul> <h3> </h3> <h3>Experimental Design: Independent Zero-Shot Runs</h3> <p>To ensure the robustness of our findings and account for the inherent stochasticity of LLM outputs, we implemented the following protocol:</p> <ol> <li> <p><strong>Zero-Shot Prompting:</strong> All experiments were conducted in a strictly <strong>zero-shot</strong> setting. No previous examples of correct bibliographies were provided in the prompt, forcing the models to rely entirely on their internal knowledge and pre-training.</p> </li> <li> <p><strong>Independent Iterations (Suffixes _1, _2, _3):</strong> Each model was prompted to generate bibliographies in <strong>three independent experimental runs</strong>. The numerical suffixes in the column names or filenames (e.g., <code>llama3.3_1</code>, <code>llama3.3_2</code>, <code>llama3.3_3</code>) correspond to these distinct iterations.</p> </li> <li> <p><strong>Independence of Data Points:</strong> Each BibTeX entry and each experimental run is treated as an independent observation. There is no memory or context carry-over between the 1st, 2nd, and 3rd runs of the same model.</p> </li> </ol> <p> </p> <p>If you use the dataset, please reference the paper:</p> <ul> <li>Picazo-Sanchez, P., & Ortiz-Martin, L. (2026). Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature. <em>Data</em>, <em>11</em>(5), 122. https://doi.org/10.3390/data11050122</li> </ul> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18657875 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | LLM-Generated Bibliography Picazo-Sanchez, Pablo Ortiz Martin, Lara <p>This dataset supports the paper <strong>'</strong><strong>Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature</strong><strong>'</strong>. It contains a collection of bibliography entries generated by LLMs to study the frequency and nature of citation hallucinations.</p> <p> </p> <p>The dataset includes bibliographic entries generated by a diverse set of Large Language Models (LLMs), covering different architectures and scales. Specifically, the following models were evaluated:</p> <ul> <li> <p><strong>Microsoft Phi Family:</strong> <code>phi:2.7b:4q</code>, <code>phi3.5:3.8b:4q</code></p> </li> <li> <p><strong>Meta Llama Family:</strong> <code>llama2:7b:4q</code>, <code>llama3.3:70b:4q</code></p> </li> <li> <p><strong>Google Gemma Family:</strong> <code>gemma:7b:4q</code>, <code>gemma2:9b:4q</code></p> </li> <li> <p><strong>Specialized/Other Models:</strong> <code>dolphin-mistral:7b:4q</code>, <code>command-r:35b:4q</code>, <code>deepSeek-r1:8b</code></p> </li> </ul> <h3> </h3> <h3>Experimental Design: Independent Zero-Shot Runs</h3> <p>To ensure the robustness of our findings and account for the inherent stochasticity of LLM outputs, we implemented the following protocol:</p> <ol> <li> <p><strong>Zero-Shot Prompting:</strong> All experiments were conducted in a strictly <strong>zero-shot</strong> setting. No previous examples of correct bibliographies were provided in the prompt, forcing the models to rely entirely on their internal knowledge and pre-training.</p> </li> <li> <p><strong>Independent Iterations (Suffixes _1, _2, _3):</strong> Each model was prompted to generate bibliographies in <strong>three independent experimental runs</strong>. The numerical suffixes in the column names or filenames (e.g., <code>llama3.3_1</code>, <code>llama3.3_2</code>, <code>llama3.3_3</code>) correspond to these distinct iterations.</p> </li> <li> <p><strong>Independence of Data Points:</strong> Each BibTeX entry and each experimental run is treated as an independent observation. There is no memory or context carry-over between the 1st, 2nd, and 3rd runs of the same model.</p> </li> </ol> <p> </p> <p>If you use the dataset, please reference the paper:</p> <ul> <li>Picazo-Sanchez, P., & Ortiz-Martin, L. (2026). Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature. <em>Data</em>, <em>11</em>(5), 122. https://doi.org/10.3390/data11050122</li> </ul> |
| title | LLM-Generated Bibliography |
| url | https://doi.org/10.5281/zenodo.18657875 |