Theoretical Proof that Auto-regressive Language Models Collapse when Real-world Data is a Finite Set

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Lecheng, Shi, Xianjie, Li, Ge, Li, Jia, Zhang, Xuanming, Dong, Yihong, Jiao, Wenpin, Mei, Hong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Self-planning Code Generation with Large Language Models
by: Jiang, Xue, et al.
Published: (2023)

Exploring Data-Efficient Adaptation of Large Language Models for Code Generation
by: Jiang, Xue, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation
by: Jiang, Xue, et al.
Published: (2024)

KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?
by: Jiang, Xue, et al.
Published: (2026)

CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
by: Zhang, Kechi, et al.
Published: (2024)

A Contradiction-Centered Model for the Emergence of Swarm Intelligence
by: Jiao, Wenpin
Published: (2025)

A Generic Model for Swarm Intelligence and Its Validations
by: Jiao, Wenpin
Published: (2017)

Large Language Model Unlearning for Source Code
by: Jiang, Xue, et al.
Published: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

Computational Thinking Reasoning in Large Language Models
by: Zhang, Kechi, et al.
Published: (2025)

Uncertainty-Guided Chain-of-Thought for Code Generation with LLMs
by: Zhu, Yuqi, et al.
Published: (2025)

aiXcoder-7B-v2: Training LLMs to Fully Utilize the Long Context in Repository-level Code Completion
by: Li, Jia, et al.
Published: (2025)

EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning
by: Liu, Huanyu, et al.
Published: (2025)

Think Anywhere in Code Generation
by: Jiang, Xue, et al.
Published: (2026)

The broken sample problem revisited: Proof of a conjecture by Bai-Hsing and high-dimensional extensions
by: Jiao, Simiao, et al.
Published: (2025)

VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications
by: Zhu, Hao, et al.
Published: (2025)

Generating Equivalent Representations of Code By A Self-Reflection Approach
by: Li, Jia, et al.
Published: (2024)

Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points
by: Zhang, Kechi, et al.
Published: (2025)

StackTrans: From Large Language Model to Large Pushdown Automata Model
by: Zhang, Kechi, et al.
Published: (2025)

Reasoning is Periodicity? Improving Large Language Models Through Effective Periodicity Modeling
by: Dong, Yihong, et al.
Published: (2025)

GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generation
by: Li, Jia, et al.
Published: (2025)

DevEval: Evaluating Code Generation in Practical Software Projects
by: Li, Jia, et al.
Published: (2024)

CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
by: Jiang, Xue, et al.
Published: (2025)

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models
by: Dong, Yihong, et al.
Published: (2024)

PACE: Improving Prompt with Actor-Critic Editing for Large Language Model
by: Dong, Yihong, et al.
Published: (2023)

Parameter Estimation in Recurrent Tumor Evolution with Finite Carrying Capacity
by: Leder, Kevin, et al.
Published: (2025)

Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
by: Tao, Yongding, et al.
Published: (2025)

Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation
by: Luo, Zhuoyan, et al.
Published: (2024)

Comparing Large Language Models and Human Programmers for Generating Programming Code
by: Wenpin Hou, et al.
Published: (2024)

The Variable Muckenhoupt Weight Revisited
by: Jia, Hongchao, et al.
Published: (2024)

EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generation
by: Tang, Jiaxiang, et al.
Published: (2024)

On the Representations of Entities in Auto-regressive Large Language Models
by: Morand, Victor, et al.
Published: (2025)

Line-level Semantic Structure Learning for Code Vulnerability Detection
by: Wang, Ziliang, et al.
Published: (2024)

SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning
by: Liu, Huanyu, et al.
Published: (2025)

Residual Feature-Reutilization Inception Network for Image Classification
by: He, Yuanpeng, et al.
Published: (2024)

ProofBridge: Auto-Formalization of Natural Language Proofs in Lean via Joint Embeddings
by: Jana, Prithwish, et al.
Published: (2025)

Self-collaboration Code Generation via ChatGPT
by: Dong, Yihong, et al.
Published: (2023)

Investigating the Impact of Data Selection Strategies on Language Model Performance
by: Gu, Jiayao, et al.
Published: (2025)