Cover Image

Reusing Overtrained Language Models Saturates Scaling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liew, Seng Pei, Kato, Takuya
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

Towards Principled Design of Mixture-of-Experts Language Models under Memory and Inference Constraints
by: Liew, Seng Pei, et al.
Published: (2026)

Test-Time Scaling Makes Overtraining Compute-Optimal
by: Roberts, Nicholas, et al.
Published: (2026)

(How) Learning Rates Regulate Catastrophic Overtraining
by: Rofin, Mark, et al.
Published: (2026)

Circuit Component Reuse Across Tasks in Transformer Language Models
by: Merullo, Jack, et al.
Published: (2023)

Method-Based Reasoning for Large Language Models: Extraction, Reuse, and Continuous Improvement
by: Su, Hong
Published: (2025)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
by: Sun, Hao, et al.
Published: (2025)

Overtrained Language Models Are Harder to Fine-Tune
by: Springer, Jacob Mitchell, et al.
Published: (2025)

Scaling Embeddings Outperforms Scaling Experts in Language Models
by: Liu, Hong, et al.
Published: (2026)

Shuffling-Aware Optimization for Private Vector Mean Estimation
by: Takagi, Shun, et al.
Published: (2026)

Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations
by: Deng, Yang, et al.
Published: (2024)

Scaling Embedding Layers in Language Models
by: Yu, Da, et al.
Published: (2025)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Scaling Laws for Multilingual Language Models
by: He, Yifei, et al.
Published: (2024)

Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixes
by: Setlur, Amrith, et al.
Published: (2026)

Towards Modular LLMs by Building and Reusing a Library of LoRAs
by: Ostapenko, Oleksiy, et al.
Published: (2024)

Scaling Beyond Masked Diffusion Language Models
by: Sahoo, Subham Sekhar, et al.
Published: (2026)

Overtrained, Not Misaligned
by: Schreiber, Joel, et al.
Published: (2026)

Benchmarking Large Language Model Uncertainty for Prompt Optimization
by: Guo, Pei-Fu, et al.
Published: (2024)

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
by: Bai, Yushi, et al.
Published: (2026)

LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts
by: Gu, Zhuohan, et al.
Published: (2024)

Resolving Discrepancies in Compute-Optimal Scaling of Language Models
by: Porian, Tomer, et al.
Published: (2024)

Scaling Laws for Discriminative Classification in Large Language Models
by: Wyatte, Dean, et al.
Published: (2024)

Emergent Abilities in Reduced-Scale Generative Language Models
by: Muckatira, Sherin, et al.
Published: (2024)

Sparse Layers are Critical to Scaling Looped Language Models
by: Lee, Ryan, et al.
Published: (2026)

Unraveling Arithmetic in Large Language Models: The Role of Algebraic Structures
by: Chang, Fu-Chieh, et al.
Published: (2024)

Benchmarks Saturate When The Model Gets Smarter Than The Judge
by: Ballon, Marthe, et al.
Published: (2026)

HEART: Emotionally-Driven Test-Time Scaling of Language Models
by: Pinto, Gabriela, et al.
Published: (2025)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Scaling Law for Language Models Training Considering Batch Size
by: Shuai, Xian, et al.
Published: (2024)

Scaling Laws for Downstream Task Performance of Large Language Models
by: Isik, Berivan, et al.
Published: (2024)

Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models
by: Koa, Kelvin J. L., et al.
Published: (2024)

Revisiting In-context Learning Inference Circuit in Large Language Models
by: Cho, Hakaze, et al.
Published: (2024)

Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
by: Li, Bolian, et al.
Published: (2026)

Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
by: Kim, Minwu, et al.
Published: (2026)

Next Semantic Scale Prediction via Hierarchical Diffusion Language Models
by: Zhou, Cai, et al.
Published: (2025)

Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces
by: Imperial, Joseph Marvin, et al.
Published: (2025)

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
by: Sardana, Nikhil, et al.
Published: (2023)

Scale Determines Whether Language Models Organize Representation Geometry for Prediction
by: Xu, Weilun
Published: (2026)

Exploring Scaling Laws for Local SGD in Large Language Model Training
by: He, Qiaozhi, et al.
Published: (2024)