Imagen de Portada

Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Filatov, Oleg, Ebert, Jan, Wang, Jiangtao, Kesselheim, Stefan
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Optimal Scaling Needs Optimal Norm
por: Filatov, Oleg, et al.
Publicado: (2025)

Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
por: Wang, Jiangtao, et al.
Publicado: (2025)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
por: Meterez, Alexandru, et al.
Publicado: (2025)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
por: Shen, Yikang, et al.
Publicado: (2024)

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size
por: Hayou, Soufiane, et al.
Publicado: (2025)

Actionable Interpretability via Causal Hypergraphs: Unravelling Batch Size Effects in Deep Learning
por: Sun, Zhongtian, et al.
Publicado: (2025)

Beware of the Batch Size: Hyperparameter Bias in Evaluating LoRA
por: Lee, Sangyoon, et al.
Publicado: (2026)

BackSlash: Rate Constrained Optimized Training of Large Language Models
por: Wu, Jun, et al.
Publicado: (2025)

Learning Rate Transfer in Normalized Transformers
por: Shigida, Boris, et al.
Publicado: (2026)

Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization
por: Palenicek, Daniel, et al.
Publicado: (2025)

Data Pruning in Generative Diffusion Models
por: Briq, Rania, et al.
Publicado: (2024)

Towards Context-Aware Domain Generalization: Understanding the Benefits and Limits of Marginal Transfer Learning
por: Müller, Jens, et al.
Publicado: (2023)

Batch Bayesian Active Learning with Partial Batch Label Sampling
por: Hu, Kangping, et al.
Publicado: (2025)

Iterative Batch Reinforcement Learning via Safe Diversified Model-based Policy Search
por: Najib, Amna, et al.
Publicado: (2024)

IT-OSE: Exploring Optimal Sample Size for Industrial Data Augmentation
por: Sun, Mingchun, et al.
Publicado: (2026)

Active Learning and Transfer Learning for Anomaly Detection in Time-Series Data
por: Kelleher, John D., et al.
Publicado: (2025)

Batch-Size Independent Regret Bounds for Combinatorial Semi-Bandits with Probabilistically Triggered Arms or Independent Arms
por: Liu, Xutong, et al.
Publicado: (2022)

Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training
por: Tyagi, Sahil, et al.
Publicado: (2026)

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration
por: Mlodozeniec, Bruno, et al.
Publicado: (2025)

How Does Critical Batch Size Scale in Pre-training?
por: Zhang, Hanlin, et al.
Publicado: (2024)

Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching
por: Wang, Meng, et al.
Publicado: (2025)

Semi-supervised Batch Learning From Logged Data
por: Aminian, Gholamali, et al.
Publicado: (2022)

Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent
por: Naganuma, Hiroki, et al.
Publicado: (2026)

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling
por: Li, Shuaipeng, et al.
Publicado: (2024)

Optimal Linear Decay Learning Rate Schedules and Further Refinements
por: Defazio, Aaron, et al.
Publicado: (2023)

BOTS: Batch Bayesian Optimization of Extended Thompson Sampling for Severely Episode-Limited RL Settings
por: Karine, Karine, et al.
Publicado: (2024)

Polynomial, trigonometric, and tropical activations
por: Khalfaoui-Hassani, Ismail, et al.
Publicado: (2025)

Extending Test-Time Scaling: A 3D Perspective with Context, Batch, and Turn
por: Yu, Chao, et al.
Publicado: (2025)

The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysis
por: Pham, Hoang, et al.
Publicado: (2025)

Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment
por: Sun, Yingchuan, et al.
Publicado: (2025)

Making Batch Normalization Great in Federated Deep Learning
por: Zhong, Jike, et al.
Publicado: (2023)

Towards Batch-to-Streaming Deep Reinforcement Learning for Continuous Control
por: De Monte, Riccardo, et al.
Publicado: (2026)

Scalable On-Policy Reinforcement Learning via Adaptive Batch Scaling
por: Park, Jongchan
Publicado: (2026)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
por: Bergsma, Shane, et al.
Publicado: (2025)

DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset
por: Liu, Weijie, et al.
Publicado: (2023)

GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data
por: Bazhenov, Gleb, et al.
Publicado: (2024)

Optimizing Data Curation through Spectral Analysis and Joint Batch Selection (SALN)
por: Sharifi, Mohammadreza
Publicado: (2024)

On the Convergence and Size Transferability of Continuous-depth Graph Neural Networks
por: Yan, Mingsong, et al.
Publicado: (2025)

Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection
por: Wu, Yinting, et al.
Publicado: (2024)

Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data
por: Zhang, Jiancheng, et al.
Publicado: (2025)