Imagen de Portada

No More Adam: Learning Rate Scaling at Initialization is All You Need

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Minghao, Xiang, Lichuan, Cai, Xu, Wen, Hongkai
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

More Agents Is All You Need
por: Li, Junyou, et al.
Publicado: (2024)

Adam-mini: Use Fewer Learning Rates To Gain More
por: Zhang, Yushun, et al.
Publicado: (2024)

Attention is All You Need Until You Need Retention
por: Yaslioglu, M. Murat
Publicado: (2025)

Cycle is All You Need: More Is Different
por: Li, Xin
Publicado: (2025)

Context is All You Need
por: Delanois, Jean Erik, et al.
Publicado: (2026)

Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learning
por: Harmel, Moritz, et al.
Publicado: (2023)

OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agents
por: Zhou, Yuhang, et al.
Publicado: (2026)

Exploitation Is All You Need... for Exploration
por: Rentschler, Micah, et al.
Publicado: (2025)

Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
por: Chen, Lingjiao, et al.
Publicado: (2024)

All You Need Is Synthetic Task Augmentation
por: Godin, Guillaume
Publicado: (2025)

Element-wise Attention Is All You Need
por: Feng, Guoxin
Publicado: (2025)

Efficient Deep Learning Board: Training Feedback Is Not All You Need
por: Gong, Lina, et al.
Publicado: (2024)

More Compute Is What You Need
por: Guo, Zhen
Publicado: (2024)

HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control
por: Zhu, Yaqiao, et al.
Publicado: (2025)

Transduction is All You Need for Structured Data Workflows
por: Gliozzo, Alfio, et al.
Publicado: (2025)

Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need
por: Wistuba, Martin, et al.
Publicado: (2024)

Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learning
por: Balloch, Jonathan C., et al.
Publicado: (2024)

Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems
por: Guo, Ping, et al.
Publicado: (2021)

Cooperation Is All You Need
por: Adeel, Ahsan, et al.
Publicado: (2023)

HDL-GPT: High-Quality HDL is All You Need
por: Kumar, Bhuvnesh, et al.
Publicado: (2024)

Capabilities Ain't All You Need: Measuring Propensities in AI
por: Romero-Alvarado, Daniel, et al.
Publicado: (2026)

Is Diversity All You Need for Scalable Robotic Manipulation?
por: Shi, Modi, et al.
Publicado: (2025)

Hedging Is Not All You Need: A Simple Baseline for Online Learning Under Haphazard Inputs
por: Buckchash, Himanshu, et al.
Publicado: (2024)

No-Regret Learning Under Adversarial Resource Constraints: A Spending Plan Is All You Need!
por: Stradi, Francesco Emanuele, et al.
Publicado: (2025)

Tensor Product Attention Is All You Need
por: Zhang, Yifan, et al.
Publicado: (2025)

Confidence Is All You Need for MI Attacks
por: Sinha, Abhishek, et al.
Publicado: (2023)

Attention Smoothing Is All You Need For Unlearning
por: Zade, Saleh Zare, et al.
Publicado: (2026)

Cross-Entropy Is All You Need To Invert the Data Generating Process
por: Reizinger, Patrik, et al.
Publicado: (2024)

TransMLA: Multi-Head Latent Attention Is All You Need
por: Meng, Fanxu, et al.
Publicado: (2025)

Context-Selective State Space Models: Feedback is All You Need
por: Zattra, Riccardo, et al.
Publicado: (2025)

Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
por: Foster, Dylan J., et al.
Publicado: (2024)

FlexControl: Computation-Aware ControlNet with Differentiable Router for Text-to-Image Generation
por: Fang, Zheng, et al.
Publicado: (2025)

Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
por: Nguyen, Tien-Phat, et al.
Publicado: (2026)

Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
por: Yu, Zhiyin, et al.
Publicado: (2026)

Dying Clusters Is All You Need -- Deep Clustering With an Unknown Number of Clusters
por: Leiber, Collin, et al.
Publicado: (2024)

The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
por: Qasim, Kaleem Ullah, et al.
Publicado: (2026)

Why Transformers Need Adam: A Hessian Perspective
por: Zhang, Yushun, et al.
Publicado: (2024)

One Masked Model is All You Need for Sensor Fault Detection, Isolation and Accommodation
por: Fu, Yiwei, et al.
Publicado: (2024)

Procedural Memory Is Not All You Need: Bridging Cognitive Gaps in LLM-Based Agents
por: Wheeler, Schaun, et al.
Publicado: (2025)

Small Graph Is All You Need: DeepStateGNN for Scalable Traffic Forecasting
por: Wölker, Yannick, et al.
Publicado: (2025)