Cover Image

Scaling Optimal LR Across Token Horizons

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bjorck, Johan, Benhaim, Alon, Chaudhary, Vishrav, Wei, Furu, Song, Xia
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Practical Analysis of Human Alignment with *PO
by: Ahrabian, Kian, et al.
Published: (2024)

Scaling Laws for Multilingual Language Models
by: He, Yifei, et al.
Published: (2024)

MathScale: Scaling Instruction Tuning for Mathematical Reasoning
by: Tang, Zhengyang, et al.
Published: (2024)

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge
by: Tang, Yao, et al.
Published: (2026)

A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia
by: Monea, Giovanni, et al.
Published: (2023)

MathPile: A Billion-Token-Scale Pretraining Corpus for Math
by: Wang, Zengzhi, et al.
Published: (2023)

Planning Transformer: Long-Horizon Offline Reinforcement Learning with Planning Tokens
by: Clinton, Joseph, et al.
Published: (2024)

Auto-ICL: In-Context Learning without Human Supervision
by: Yang, Jinghan, et al.
Published: (2023)

Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers
by: Yang, Wang, et al.
Published: (2026)

OptScale: Probabilistic Optimality for Inference-time Scaling
by: Wang, Youkang, et al.
Published: (2025)

Synthetic Computers at Scale for Long-Horizon Productivity Simulation
by: Ge, Tao, et al.
Published: (2026)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
by: Kwek, Eugene, et al.
Published: (2025)

MarkovScale: Towards Optimal Sequential Scaling at Inference Time
by: Wang, Youkang, et al.
Published: (2026)

Textual Aesthetics in Large Language Models
by: Jiang, Lingjie, et al.
Published: (2024)

Multi-Head Mixture-of-Experts
by: Wu, Xun, et al.
Published: (2024)

Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
by: Li, Meng, et al.
Published: (2025)

POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization
by: Karaman, Batuhan K., et al.
Published: (2024)

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models
by: Raj, Bharath, et al.
Published: (2024)

Generating Pretraining Tokens from Organic Data for Data-Bound Scaling
by: Yu, Zichun, et al.
Published: (2026)

Learning to Route LLMs with Confidence Tokens
by: Chuang, Yu-Neng, et al.
Published: (2024)

GRIN: GRadient-INformed MoE
by: Liu, Liyuan, et al.
Published: (2024)

On The Adaptation of Unlimiformer for Decoder-Only Transformers
by: Ahrabian, Kian, et al.
Published: (2024)

Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards
by: Shen, Yiran, et al.
Published: (2025)

SafetyNet: Detecting Harmful Outputs in LLMs by Modeling and Monitoring Deceptive Behaviors
by: Chaudhary, Maheep, et al.
Published: (2025)

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies
by: Ovalle, Anaelia, et al.
Published: (2023)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
by: Miyamoto, Sora, et al.
Published: (2026)

Are More Tokens Rational? Inference-Time Scaling in Language Models as Adaptive Resource Rationality
by: Hu, Zhimin, et al.
Published: (2026)

In-context Autoencoder for Context Compression in a Large Language Model
by: Ge, Tao, et al.
Published: (2023)

Compute Optimal Scaling of Skills: Knowledge vs Reasoning
by: Roberts, Nicholas, et al.
Published: (2025)

TokenButler: Token Importance is Predictable
by: Akhauri, Yash, et al.
Published: (2025)

Implicit Strategic Optimization: Rethinking Long-Horizon Decision-Making in Adversarial Poker Environments
by: Xia, Boyang, et al.
Published: (2026)

Latency and Token-Aware Test-Time Compute
by: Huang, Jenny Y., et al.
Published: (2025)

Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
by: Mahabadi, Rabeeh Karimi, et al.
Published: (2025)

SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
by: Vejendla, Harshil
Published: (2025)

Lossless Token Sequence Compression via Meta-Tokens
by: Harvill, John, et al.
Published: (2025)

Contrastive Token-level Explanations for Graph-based Rumour Detection
by: Chin, Daniel Wai Kit, et al.
Published: (2025)

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
by: Shao, Rulin, et al.
Published: (2024)

Adversarial Tokenization
by: Geh, Renato Lui, et al.
Published: (2025)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

Generative Representational Instruction Tuning
by: Muennighoff, Niklas, et al.
Published: (2024)