Cover Image

Saved in:
Bibliographic Details
Main Author: Craig, Gordon
Format: Recurso digital
Language:English
Published: Zenodo 2026
Online Access:https://doi.org/10.5281/zenodo.19367539
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866901113487228928
author Craig, Gordon
author_facet Craig, Gordon
contents <h1><span>NTT: Numerical Turing Test -- Data Archive</span></h1> <h2><span>Overview</span></h2> <p><span>This archive contains the data, prompts, and results for the Numerical Turing Test (NTT) study, which evaluates whether large language models can generate financial transaction amounts that are statistically indistinguishable from real human data.</span></p> <h2><span>Contents</span></h2> <h3><span>data/</span></h3> <p><span>Raw LLM-generated outputs and real reference datasets:</span></p> <ul> <li> <p><span><strong><span>LLM outputs</span></strong></span><span>: Transaction amounts generated by Claude, ChatGPT, Gemini, and Grok under 15+ prompt conditions (A through M3), with multiple replicates per condition.</span></p> </li> <li> <p><span><strong><span>Reference data</span></strong></span><span>: Real financial transaction amounts from public datasets (e.g., NYC taxi tips) used as ground truth for statistical comparison.</span></p> </li> <li> <p><span><strong><span>Cross-currency outputs</span></strong></span><span>: LLM-generated amounts in GBP, EUR, JPY, INR, and CNY.</span></p> </li> </ul> <h3><span>prompts/</span></h3> <p><span>Exact prompt text used for each experimental condition. See </span><span><code>prompts/README.md</code></span><span> for the full condition taxonomy (baseline, coached, adversarial, chain-of-thought, few-shot, iterative, decomposition, and code generation).</span></p> <h3><span>results/</span></h3> <p><span>Pre-computed analysis results including:</span></p> <ul> <li> <p><span>Benford's Law analysis (first and second digit)</span></p> </li> <li> <p><span>Round-number frequency analysis</span></p> </li> <li> <p><span>Value repetition and serial correlation tests</span></p> </li> <li> <p><span>CBAD compression-based analysis</span></p> </li> <li> <p><span>Prime factorization encoding metrics</span></p> </li> <li> <p><span>Aggregate NTT scores and classification results</span></p> </li> </ul> <h2><span>Forensic Test Battery</span></h2> <p><span>Each sample (real or LLM-generated) is evaluated on:</span></p> <ol> <li> <p><span><strong><span>Benford's Law</span></strong></span><span> -- First-digit and second-digit distribution vs. expected log frequencies</span></p> </li> <li> <p><span><strong><span>Round-number rates</span></strong></span><span> -- Frequency of amounts ending in .00, .50, .25, .75</span></p> </li> <li> <p><span><strong><span>Value repetition</span></strong></span><span> -- Duplicate and near-duplicate detection</span></p> </li> <li> <p><span><strong><span>Serial correlation</span></strong></span><span> -- Lag-1 autocorrelation in amount sequences</span></p> </li> <li> <p><span><strong><span>Prime factorization analysis (CBAD)</span></strong></span><span> -- Compression-based anomaly detection on factor-encoded cent values</span></p> </li> </ol> <h2><span>File Formats</span></h2> <ul> <li> <p><span><code>.csv</code></span><span> -- Tabular data (amounts, results, metadata)</span></p> </li> <li> <p><span><code>.txt</code></span><span> -- Raw LLM output (one amount per line) and prompt text</span></p> </li> <li> <p><span><code>.json</code></span><span> -- Structured results and configuration</span></p> </li> </ul> <h2><span>Citation</span></h2> <p><span>If you use this data, please cite:</span></p> <pre><span>@article{gordon2025ntt,</span><br><span> title={The Numerical Turing Test: Can LLMs Generate Statistically Human Financial Data?},</span><br><span> author={Gordon, Craig S.},</span><br><span> year={2026}</span><br><span>}</span></pre> <h2><span>License</span></h2> <p><span>See LICENSE file for terms.</span></p>
format Recurso digital
id zenodo_https___doi_org_10_5281_zenodo_19367539
institution Zenodo
language eng
publishDate 2026
publisher Zenodo
record_format zenodo
spellingShingle Numerical Turing Test
Craig, Gordon
<h1><span>NTT: Numerical Turing Test -- Data Archive</span></h1> <h2><span>Overview</span></h2> <p><span>This archive contains the data, prompts, and results for the Numerical Turing Test (NTT) study, which evaluates whether large language models can generate financial transaction amounts that are statistically indistinguishable from real human data.</span></p> <h2><span>Contents</span></h2> <h3><span>data/</span></h3> <p><span>Raw LLM-generated outputs and real reference datasets:</span></p> <ul> <li> <p><span><strong><span>LLM outputs</span></strong></span><span>: Transaction amounts generated by Claude, ChatGPT, Gemini, and Grok under 15+ prompt conditions (A through M3), with multiple replicates per condition.</span></p> </li> <li> <p><span><strong><span>Reference data</span></strong></span><span>: Real financial transaction amounts from public datasets (e.g., NYC taxi tips) used as ground truth for statistical comparison.</span></p> </li> <li> <p><span><strong><span>Cross-currency outputs</span></strong></span><span>: LLM-generated amounts in GBP, EUR, JPY, INR, and CNY.</span></p> </li> </ul> <h3><span>prompts/</span></h3> <p><span>Exact prompt text used for each experimental condition. See </span><span><code>prompts/README.md</code></span><span> for the full condition taxonomy (baseline, coached, adversarial, chain-of-thought, few-shot, iterative, decomposition, and code generation).</span></p> <h3><span>results/</span></h3> <p><span>Pre-computed analysis results including:</span></p> <ul> <li> <p><span>Benford's Law analysis (first and second digit)</span></p> </li> <li> <p><span>Round-number frequency analysis</span></p> </li> <li> <p><span>Value repetition and serial correlation tests</span></p> </li> <li> <p><span>CBAD compression-based analysis</span></p> </li> <li> <p><span>Prime factorization encoding metrics</span></p> </li> <li> <p><span>Aggregate NTT scores and classification results</span></p> </li> </ul> <h2><span>Forensic Test Battery</span></h2> <p><span>Each sample (real or LLM-generated) is evaluated on:</span></p> <ol> <li> <p><span><strong><span>Benford's Law</span></strong></span><span> -- First-digit and second-digit distribution vs. expected log frequencies</span></p> </li> <li> <p><span><strong><span>Round-number rates</span></strong></span><span> -- Frequency of amounts ending in .00, .50, .25, .75</span></p> </li> <li> <p><span><strong><span>Value repetition</span></strong></span><span> -- Duplicate and near-duplicate detection</span></p> </li> <li> <p><span><strong><span>Serial correlation</span></strong></span><span> -- Lag-1 autocorrelation in amount sequences</span></p> </li> <li> <p><span><strong><span>Prime factorization analysis (CBAD)</span></strong></span><span> -- Compression-based anomaly detection on factor-encoded cent values</span></p> </li> </ol> <h2><span>File Formats</span></h2> <ul> <li> <p><span><code>.csv</code></span><span> -- Tabular data (amounts, results, metadata)</span></p> </li> <li> <p><span><code>.txt</code></span><span> -- Raw LLM output (one amount per line) and prompt text</span></p> </li> <li> <p><span><code>.json</code></span><span> -- Structured results and configuration</span></p> </li> </ul> <h2><span>Citation</span></h2> <p><span>If you use this data, please cite:</span></p> <pre><span>@article{gordon2025ntt,</span><br><span> title={The Numerical Turing Test: Can LLMs Generate Statistically Human Financial Data?},</span><br><span> author={Gordon, Craig S.},</span><br><span> year={2026}</span><br><span>}</span></pre> <h2><span>License</span></h2> <p><span>See LICENSE file for terms.</span></p>
title Numerical Turing Test
url https://doi.org/10.5281/zenodo.19367539

Similar Items