Cover Image

Scaling and Transferability of Annealing Strategies in Large Language Model Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Siqi, Chen, Zhengyu, Xiao, Teng, Lv, Zheqi, Yang, Jinluan, Cai, Xunliang, Wang, Jingang, Li, Xiaomeng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
by: Chen, Zhengyu, et al.
Published: (2025)

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models
by: Wang, Siqi, et al.
Published: (2024)

What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
by: Gong, Zhuocheng, et al.
Published: (2024)

Leveraging Invariant Principle for Heterophilic Graph Structure Distribution Shifts
by: Yang, Jinluan, et al.
Published: (2024)

Unifying Adversarial Perturbation for Graph Neural Networks
by: Yang, Jinluan, et al.
Published: (2025)

A Rolling Stone Gathers No Moss: Adaptive Policy Optimization for Stable Self-Evaluation in Large Multimodal Models
by: Wang, Wenkai, et al.
Published: (2025)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
by: Wang, Chao, et al.
Published: (2026)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Intelligent Model Update Strategy for Sequential Recommendation
by: Lv, Zheqi, et al.
Published: (2023)

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
by: Diao, Muxi, et al.
Published: (2024)

Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspective
by: Kong, Deyang, et al.
Published: (2025)

Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy
by: Zhang, Xiaoyun, et al.
Published: (2025)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
by: Yang, Jing, et al.
Published: (2025)

Annealing Self-Distillation Rectification Improves Adversarial Training
by: Wu, Yu-Yu, et al.
Published: (2023)

Scaling Embeddings Outperforms Scaling Experts in Language Models
by: Liu, Hong, et al.
Published: (2026)

Scaling Law with Learning Rate Annealing
by: Tissue, Howe, et al.
Published: (2024)

Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models
by: Sun, Shaoning, et al.
Published: (2026)

Enhancing Hepatopathy Clinical Trial Efficiency: A Secure, Large Language Model-Powered Pre-Screening Pipeline
by: Gui, Xiongbin, et al.
Published: (2025)

Test-Time Training on Graphs with Large Language Models (LLMs)
by: Zhang, Jiaxin, et al.
Published: (2024)

PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
by: Xiao, He, et al.
Published: (2025)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
by: Jia, Sheng, et al.
Published: (2025)

Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap
by: Sun, Yifan, et al.
Published: (2025)

Efficient Cross-Architecture Knowledge Transfer for Large-Scale Online User Response Prediction
by: Wu, Yucheng, et al.
Published: (2026)

ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models
by: Liu, Jincheng, et al.
Published: (2025)

ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation
by: Tang, Zihao, et al.
Published: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
by: Zhang, Jun, et al.
Published: (2025)

Towards a Unified View of Large Language Model Post-Training
by: Lv, Xingtai, et al.
Published: (2025)

Estimating the Effects of Sample Training Orders for Large Language Models without Retraining
by: Yang, Hao, et al.
Published: (2025)

LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
by: Lee, Jung Hyun, et al.
Published: (2024)

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning
by: Wang, Yejie, et al.
Published: (2024)

Policy and World Modeling Co-Training for Language Agents
by: Lu, Ning, et al.
Published: (2026)

Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach
by: Mao, Yuzhu, et al.
Published: (2024)

Large Scale Transfer Learning for Tabular Data via Language Modeling
by: Gardner, Josh, et al.
Published: (2024)

Training Language Models with Language Feedback at Scale
by: Scheurer, Jérémy, et al.
Published: (2023)

Post Training Quantization of Large Language Models with Microscaling Formats
by: Sharify, Sayeh, et al.
Published: (2024)

Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds
by: Wang, Fan, et al.
Published: (2025)

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
by: Wu, Boqian, et al.
Published: (2026)

TabICL: A Tabular Foundation Model for In-Context Learning on Large Data
by: Qu, Jingang, et al.
Published: (2025)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
by: Xi, Zhiheng, et al.
Published: (2024)

Large Language Models as Tool Makers
by: Cai, Tianle, et al.
Published: (2023)