Cover Image

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Kairong, Wen, Haodong, Hu, Shengding, Sun, Zhenbo, Liu, Zhiyuan, Sun, Maosong, Lyu, Kaifeng, Chen, Wenguang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
by: Luo, Kairong, et al.
Published: (2025)

States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly
by: Chen, Junhao, et al.
Published: (2024)

Larger Datasets Can Be Repeated More: A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression
by: Yan, Tingkai, et al.
Published: (2025)

Stuffed Mamba: Oversized States Lead to the Inability to Forget
by: Chen, Yingfa, et al.
Published: (2024)

PCMind-2.1-Kaiyuan-2B Technical Report
by: Luo, Kairong, et al.
Published: (2025)

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models
by: Zhao, Ranchi, et al.
Published: (2024)

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models
by: Zhang, Xinrong, et al.
Published: (2024)

The Power of Power Law: Asymmetry Enables Compositional Reasoning
by: Wang, Zixuan, et al.
Published: (2026)

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs
by: He, Chaoqun, et al.
Published: (2024)

Predicting Emergent Abilities with Infinite Resolution Evaluation
by: Hu, Shengding, et al.
Published: (2023)

Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
by: Huang, Yufei, et al.
Published: (2024)

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
by: Zhang, Xinrong, et al.
Published: (2024)

LEGENT: Open Platform for Embodied Agents
by: Cheng, Zhili, et al.
Published: (2024)

Label Dependencies-aware Set Prediction Networks for Multi-label Text Classification
by: Xinkai, Du, et al.
Published: (2023)

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
by: Chen, Weize, et al.
Published: (2024)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
by: He, Chaoqun, et al.
Published: (2024)

Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slip Scripts
by: Chen, Yingfa, et al.
Published: (2024)

Representation Learning for Natural Language Processing
by: Liu, Zhiyuan, et al.
Published: (2020)

Densing Law of LLMs
by: Xiao, Chaojun, et al.
Published: (2024)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)

RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval
by: Wen, Kaiyue, et al.
Published: (2024)

Personality-affected Emotion Generation in Dialog Systems
by: Wen, Zhiyuan, et al.
Published: (2024)

KG-Infused RAG: Augmenting Corpus-Based RAG with External Knowledge Graphs
by: Wu, Dingjun, et al.
Published: (2025)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
by: Wen, Kaiyue, et al.
Published: (2024)

VisRAG 2.0: Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation
by: Sun, Yubo, et al.
Published: (2025)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
by: Shen, Yikang, et al.
Published: (2024)

From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora
by: Shen, Yingli, et al.
Published: (2025)

EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents
by: Cheng, Zhili, et al.
Published: (2025)

CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
by: Zhao, Weilin, et al.
Published: (2023)

The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE Training
by: Chen, Weize, et al.
Published: (2025)

Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold
by: Li, Xinghan, et al.
Published: (2025)

H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
by: Gao, Cheng, et al.
Published: (2025)

Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate
by: Bu, Zhiqi, et al.
Published: (2026)

Loss-to-Loss Prediction: Scaling Laws for All Datasets
by: Brandfonbrener, David, et al.
Published: (2024)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection
by: Huang, Yufei, et al.
Published: (2024)

CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning
by: Li, Ran, et al.
Published: (2026)

Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning
by: Wang, Xiaorong, et al.
Published: (2025)

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
by: Guo, Yiju, et al.
Published: (2024)