Imagen de Portada

AdaLRS: Loss-Guided Adaptive Learning Rate Search for Efficient Foundation Model Pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Dong, Hongyuan, Yang, Dingkang, Liang, Xiao, Feng, Chao, Ran, Jiao
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
por: Lv, Kai, et al.
Publicado: (2023)

Scalable Vision Language Model Training via High Quality Data Curation
por: Dong, Hongyuan, et al.
Publicado: (2025)

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling
por: Miao, Yongliang, et al.
Publicado: (2026)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
por: Chang, Aofei, et al.
Publicado: (2024)

AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation
por: Du, Weihua, et al.
Publicado: (2026)

AdaFRUGAL: Adaptive Memory-Efficient Training with Dynamic Control
por: Bui, Quang-Hung, et al.
Publicado: (2025)

An Integrated Data Processing Framework for Pretraining Foundation Models
por: Sun, Yiding, et al.
Publicado: (2024)

AdaSplash: Adaptive Sparse Flash Attention
por: Gonçalves, Nuno, et al.
Publicado: (2025)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
por: Zhao, Bowen, et al.
Publicado: (2024)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
por: Yang, Yifan, et al.
Publicado: (2024)

AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting
por: Li, Renda, et al.
Publicado: (2025)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
por: Hu, Yuezhou, et al.
Publicado: (2025)

Parameter-Efficient Fine-Tuning for Foundation Models
por: Zhang, Dan, et al.
Publicado: (2025)

DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding
por: Jiang, Yue, et al.
Publicado: (2025)

Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives
por: Xiong, Wei, et al.
Publicado: (2025)

AdaBoN: Adaptive Best-of-N Alignment
por: Raman, Vinod, et al.
Publicado: (2025)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
por: Wen, Kaiyue, et al.
Publicado: (2024)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
por: Chen, Shaolong, et al.
Publicado: (2026)

AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
por: Zhou, Hongyi, et al.
Publicado: (2025)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
por: Zhou, Zhanhui, et al.
Publicado: (2024)

Towards Efficient Active Learning in NLP via Pretrained Representations
por: Vysogorets, Artem, et al.
Publicado: (2024)

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
por: Grangier, David, et al.
Publicado: (2024)

UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science
por: Yang, Yazheng, et al.
Publicado: (2023)

SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature Refinement
por: Yin, Weijie, et al.
Publicado: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
por: Feng, Jiahai, et al.
Publicado: (2024)

ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining
por: Bal, Melis Ilayda, et al.
Publicado: (2025)

BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search
por: Sun, Linzhuang, et al.
Publicado: (2024)

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
por: Yu, Zichun, et al.
Publicado: (2024)

Efficient and Flexible Topic Modeling using Pretrained Embeddings and Bag of Sentences
por: Schneider, Johannes
Publicado: (2023)

ixi-GEN: Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining
por: Kim, Seonwu, et al.
Publicado: (2025)

AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners
por: Koh, Woosung, et al.
Publicado: (2025)

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding
por: Feng, Qi, et al.
Publicado: (2025)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
por: Ma, Xuezhe, et al.
Publicado: (2024)

Value-Guided Search for Efficient Chain-of-Thought Reasoning
por: Wang, Kaiwen, et al.
Publicado: (2025)

Group-Level Data Selection for Efficient Pretraining
por: Yu, Zichun, et al.
Publicado: (2025)

Efficient Stagewise Pretraining via Progressive Subnetworks
por: Panigrahi, Abhishek, et al.
Publicado: (2024)

Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
por: Zhou, Xuwen, et al.
Publicado: (2026)

Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation
por: Arias, Esteban Garces, et al.
Publicado: (2024)

AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
por: Lu, Kuan-Wei, et al.
Publicado: (2025)

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
por: Luo, Kairong, et al.
Publicado: (2025)