Image de couverture de livre

Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Barron, Joshua, White, Devin
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Too Big to Fool: Resisting Deception in Language Models
par: Samsami, Mohammad Reza, et autres
Publié: (2024)

Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
par: Bergner, Benjamin, et autres
Publié: (2024)

Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers
par: Pelofske, Elijah, et autres
Publié: (2024)

Impact of Layer Norm on Memorization and Generalization in Transformers
par: Singhal, Rishi, et autres
Publié: (2025)

Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in Product QA Agents
par: Lewis, Ashley, et autres
Publié: (2025)

Thinking Deeper, Not Longer: Depth-Recurrent Transformers for Compositional Generalization
par: Chen, Hung-Hsuan
Publié: (2026)

Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
par: Kohli, Harsh, et autres
Publié: (2026)

Memorization in In-Context Learning
par: Golchin, Shahriar, et autres
Publié: (2024)

To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
par: Kothapalli, Vignesh, et autres
Publié: (2025)

Post-edits Are Preferences Too
par: Berger, Nathaniel, et autres
Publié: (2024)

Mitigating Memorization In Language Models
par: Sakarvadia, Mansi, et autres
Publié: (2024)

Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models
par: Patel, Laksh, et autres
Publié: (2025)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
par: Wang, Xinyi, et autres
Publié: (2024)

Memorization: A Close Look at Books
par: Ma, Iris, et autres
Publié: (2025)

Titans: Learning to Memorize at Test Time
par: Behrouz, Ali, et autres
Publié: (2024)

Memorization Dynamics of Fill-in-the-Middle Pretraining
par: von Arx, Tobias, et autres
Publié: (2026)

Detecting Memorization in Large Language Models
par: Slonski, Eduardo
Publié: (2024)

Memorization-Compression Cycles Improve Generalization
par: Yu, Fangyuan
Publié: (2025)

Reinforcement Learning on Pre-Training Data
par: Li, Siheng, et autres
Publié: (2025)

Automated Software Vulnerability Static Code Analysis Using Generative Pre-Trained Transformer Models
par: Pelofske, Elijah, et autres
Publié: (2024)

Exploring Memorization in Fine-tuned Language Models
par: Zeng, Shenglai, et autres
Publié: (2023)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
par: Lee, Philip Heejun
Publié: (2025)

Train Long, Think Short: Curriculum Learning for Efficient Reasoning
par: Hammoud, Hasan Abed Al Kader, et autres
Publié: (2025)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Think Before You Act: Decision Transformers with Working Memory
par: Kang, Jikun, et autres
Publié: (2023)

Memorization vs. Reasoning: Updating LLMs with New Knowledge
par: Li, Aochong Oliver, et autres
Publié: (2025)

Skewed Memorization in Large Language Models: Quantification and Decomposition
par: Li, Hao, et autres
Publié: (2025)

An Evaluation on Large Language Model Outputs: Discourse and Memorization
par: de Wynter, Adrian, et autres
Publié: (2023)

To Memorize or to Retrieve: Scaling Laws for RAG-Considerate Pretraining
par: Singh, Karan, et autres
Publié: (2026)

Think before you speak: Training Language Models With Pause Tokens
par: Goyal, Sachin, et autres
Publié: (2023)

Pre-training a Transformer-Based Generative Model Using a Small Sepedi Dataset
par: Ramalepe, Simon P., et autres
Publié: (2025)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
par: Bossy, Thierry, et autres
Publié: (2025)

Pruning as a Defense: Reducing Memorization in Large Language Models
par: Gupta, Mansi, et autres
Publié: (2025)

Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
par: Szep, Marton, et autres
Publié: (2026)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
par: Schröder, Christopher, et autres
Publié: (2024)

Learning Dynamics in Continual Pre-Training for Large Language Models
par: Wang, Xingjin, et autres
Publié: (2025)

Direct-Scoring NLG Evaluators Can Use Pairwise Comparisons Too
par: Lawrence, Logan, et autres
Publié: (2025)

Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
par: Wu, Mingqi, et autres
Publié: (2025)

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction
par: Ma, Mingyu Derek, et autres
Publié: (2025)

Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion
par: Zaman, Kerem, et autres
Publié: (2023)