Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Author:	Craig, Gordon
Format:	Recurso digital
Language:	English
Published:	Zenodo 2026
Online Access:	https://doi.org/10.5281/zenodo.19367539
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

<h1>NTT: Numerical Turing Test -- Data Archive</h1> <h2>Overview</h2> This archive contains the data, prompts, and results for the Numerical Turing Test (NTT) study, which evaluates whether large language models can generate financial transaction amounts that are statistically indistinguishable from real human data. <h2>Contents</h2> <h3>data/</h3> Raw LLM-generated outputs and real reference datasets: <ul> <li> LLM outputs: Transaction amounts generated by Claude, ChatGPT, Gemini, and Grok under 15+ prompt conditions (A through M3), with multiple replicates per condition. </li> <li> Reference data: Real financial transaction amounts from public datasets (e.g., NYC taxi tips) used as ground truth for statistical comparison. </li> <li> Cross-currency outputs: LLM-generated amounts in GBP, EUR, JPY, INR, and CNY. </li> </ul> <h3>prompts/</h3> Exact prompt text used for each experimental condition. See <code>prompts/README.md</code> for the full condition taxonomy (baseline, coached, adversarial, chain-of-thought, few-shot, iterative, decomposition, and code generation). <h3>results/</h3> Pre-computed analysis results including: <ul> <li> Benford's Law analysis (first and second digit) </li> <li> Round-number frequency analysis </li> <li> Value repetition and serial correlation tests </li> <li> CBAD compression-based analysis </li> <li> Prime factorization encoding metrics </li> <li> Aggregate NTT scores and classification results </li> </ul> <h2>Forensic Test Battery</h2> Each sample (real or LLM-generated) is evaluated on: <ol> <li> Benford's Law -- First-digit and second-digit distribution vs. expected log frequencies </li> <li> Round-number rates -- Frequency of amounts ending in .00, .50, .25, .75 </li> <li> Value repetition -- Duplicate and near-duplicate detection </li> <li> Serial correlation -- Lag-1 autocorrelation in amount sequences </li> <li> Prime factorization analysis (CBAD) -- Compression-based anomaly detection on factor-encoded cent values </li> </ol> <h2>File Formats</h2> <ul> <li> <code>.csv</code> -- Tabular data (amounts, results, metadata) </li> <li> <code>.txt</code> -- Raw LLM output (one amount per line) and prompt text </li> <li> <code>.json</code> -- Structured results and configuration </li> </ul> <h2>Citation</h2> If you use this data, please cite: <pre>@article{gordon2025ntt, title={The Numerical Turing Test: Can LLMs Generate Statistically Human Financial Data?}, author={Gordon, Craig S.}, year={2026} }</pre> <h2>License</h2> See LICENSE file for terms.

Similar Items