Copertina

Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Barron, Joshua, White, Devin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Too Big to Fool: Resisting Deception in Language Models
di: Samsami, Mohammad Reza, et al.
Pubblicazione: (2024)

Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
di: Bergner, Benjamin, et al.
Pubblicazione: (2024)

Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers
di: Pelofske, Elijah, et al.
Pubblicazione: (2024)

Impact of Layer Norm on Memorization and Generalization in Transformers
di: Singhal, Rishi, et al.
Pubblicazione: (2025)

Winning Big with Small Models: Knowledge Distillation vs. Self-Training for Reducing Hallucination in Product QA Agents
di: Lewis, Ashley, et al.
Pubblicazione: (2025)

Thinking Deeper, Not Longer: Depth-Recurrent Transformers for Compositional Generalization
di: Chen, Hung-Hsuan
Pubblicazione: (2026)

Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
di: Kohli, Harsh, et al.
Pubblicazione: (2026)

Memorization in In-Context Learning
di: Golchin, Shahriar, et al.
Pubblicazione: (2024)

To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
di: Kothapalli, Vignesh, et al.
Pubblicazione: (2025)

Post-edits Are Preferences Too
di: Berger, Nathaniel, et al.
Pubblicazione: (2024)

Mitigating Memorization In Language Models
di: Sakarvadia, Mansi, et al.
Pubblicazione: (2024)

Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models
di: Patel, Laksh, et al.
Pubblicazione: (2025)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
di: Wang, Xinyi, et al.
Pubblicazione: (2024)

Memorization: A Close Look at Books
di: Ma, Iris, et al.
Pubblicazione: (2025)

Titans: Learning to Memorize at Test Time
di: Behrouz, Ali, et al.
Pubblicazione: (2024)

Memorization Dynamics of Fill-in-the-Middle Pretraining
di: von Arx, Tobias, et al.
Pubblicazione: (2026)

Detecting Memorization in Large Language Models
di: Slonski, Eduardo
Pubblicazione: (2024)

Memorization-Compression Cycles Improve Generalization
di: Yu, Fangyuan
Pubblicazione: (2025)

Reinforcement Learning on Pre-Training Data
di: Li, Siheng, et al.
Pubblicazione: (2025)

Automated Software Vulnerability Static Code Analysis Using Generative Pre-Trained Transformer Models
di: Pelofske, Elijah, et al.
Pubblicazione: (2024)

Exploring Memorization in Fine-tuned Language Models
di: Zeng, Shenglai, et al.
Pubblicazione: (2023)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
di: Lee, Philip Heejun
Pubblicazione: (2025)

Train Long, Think Short: Curriculum Learning for Efficient Reasoning
di: Hammoud, Hasan Abed Al Kader, et al.
Pubblicazione: (2025)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
di: Hu, Jerry Yao-Chieh, et al.
Pubblicazione: (2024)

Think Before You Act: Decision Transformers with Working Memory
di: Kang, Jikun, et al.
Pubblicazione: (2023)

Memorization vs. Reasoning: Updating LLMs with New Knowledge
di: Li, Aochong Oliver, et al.
Pubblicazione: (2025)

Skewed Memorization in Large Language Models: Quantification and Decomposition
di: Li, Hao, et al.
Pubblicazione: (2025)

An Evaluation on Large Language Model Outputs: Discourse and Memorization
di: de Wynter, Adrian, et al.
Pubblicazione: (2023)

To Memorize or to Retrieve: Scaling Laws for RAG-Considerate Pretraining
di: Singh, Karan, et al.
Pubblicazione: (2026)

Think before you speak: Training Language Models With Pause Tokens
di: Goyal, Sachin, et al.
Pubblicazione: (2023)

Pre-training a Transformer-Based Generative Model Using a Small Sepedi Dataset
di: Ramalepe, Simon P., et al.
Pubblicazione: (2025)

Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
di: Bossy, Thierry, et al.
Pubblicazione: (2025)

Pruning as a Defense: Reducing Memorization in Large Language Models
di: Gupta, Mansi, et al.
Pubblicazione: (2025)

Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
di: Szep, Marton, et al.
Pubblicazione: (2026)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
di: Schröder, Christopher, et al.
Pubblicazione: (2024)

Learning Dynamics in Continual Pre-Training for Large Language Models
di: Wang, Xingjin, et al.
Pubblicazione: (2025)

Direct-Scoring NLG Evaluators Can Use Pairwise Comparisons Too
di: Lawrence, Logan, et al.
Pubblicazione: (2025)

Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
di: Wu, Mingqi, et al.
Pubblicazione: (2025)

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction
di: Ma, Mingyu Derek, et al.
Pubblicazione: (2025)

Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion
di: Zaman, Kerem, et al.
Pubblicazione: (2023)