Imagen de Portada

Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspective

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Mengfan, Zheng, Da, Su, Junwei, Wu, Chuan
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
por: Ostroukhov, Petr, et al.
Publicado: (2024)

Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch
por: Bajaj, Saurabh, et al.
Publicado: (2024)

Scaling Law for Language Models Training Considering Batch Size
por: Shuai, Xian, et al.
Publicado: (2024)

Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training
por: Merrill, William, et al.
Publicado: (2025)

Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling
por: Li, Shuaipeng, et al.
Publicado: (2024)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
por: Meterez, Alexandru, et al.
Publicado: (2025)

Convergence Bound and Critical Batch Size of Muon Optimizer
por: Sato, Naoki, et al.
Publicado: (2025)

DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset
por: Liu, Weijie, et al.
Publicado: (2023)

DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation
por: Chen, Yuen, et al.
Publicado: (2025)

Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
por: Lau, Tim Tsz-Kit, et al.
Publicado: (2024)

Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference
por: Rathi, Saksham, et al.
Publicado: (2026)

Collaborative Batch Size Optimization for Federated Learning
por: Geimer, Arno, et al.
Publicado: (2025)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
por: Islamov, Rustem, et al.
Publicado: (2026)

BG-HGNN: Toward Efficient Learning for Complex Heterogeneous Graphs
por: Su, Junwei, et al.
Publicado: (2024)

Smaller Batches, Bigger Gains? Investigating the Impact of Batch Sizes on Reinforcement Learning Based Real-World Production Scheduling
por: Müller, Arthur, et al.
Publicado: (2024)

How Does Critical Batch Size Scale in Pre-training?
por: Zhang, Hanlin, et al.
Publicado: (2024)

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
por: Kamo, Keisuke, et al.
Publicado: (2025)

Beware of the Batch Size: Hyperparameter Bias in Evaluating LoRA
por: Lee, Sangyoon, et al.
Publicado: (2026)

A Non-Asymptotic Convergent Analysis for Scored-Based Graph Generative Model via a System of Stochastic Differential Equations
por: Su, Junwei, et al.
Publicado: (2025)

Efficient GNN Training Through Structure-Aware Randomized Mini-Batching
por: Balaji, Vignesh, et al.
Publicado: (2025)

Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis
por: Kondo, Yuichi, et al.
Publicado: (2025)

On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
por: Su, Junwei, et al.
Publicado: (2025)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
por: Lau, Tim Tsz-Kit, et al.
Publicado: (2024)

Adaptive Batch Size for Privately Finding Second-Order Stationary Points
por: Liu, Daogao, et al.
Publicado: (2024)

Debiasing Mini-Batch Quadratics for Applications in Deep Learning
por: Tatzel, Lukas, et al.
Publicado: (2024)

Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
por: Lau, Tim Tsz-Kit, et al.
Publicado: (2024)

Mini-Batch Kernel $k$-means
por: Jourdan, Ben, et al.
Publicado: (2024)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
por: Oowada, Kanata, et al.
Publicado: (2025)

Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
por: Marek, Martin, et al.
Publicado: (2025)

On the Topology Awareness and Generalization Performance of Graph Neural Networks
por: Su, Junwei, et al.
Publicado: (2024)

Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
por: Garg, Sachin, et al.
Publicado: (2026)

Mini-Batch Class Composition Bias in Link Prediction
por: Maguire, Kieran, et al.
Publicado: (2026)

The Effect of Batch Size on Contrastive Self-Supervised Speech Representation Learning
por: Vaessen, Nik, et al.
Publicado: (2024)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
por: Umeda, Hikaru, et al.
Publicado: (2024)

Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit
por: Filatov, Oleg, et al.
Publicado: (2024)

Optimal Batch-Size Control for Low-Latency Federated Learning with Device Heterogeneity
por: Yang, Huiling, et al.
Publicado: (2025)

Scalable Graph Attention-based Instance Selection via Mini-Batch Sampling and Hierarchical Hashing
por: Rustamov, Zahiriddin, et al.
Publicado: (2025)

Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance
por: Piao, XinYu, et al.
Publicado: (2021)

One Size Does Not Fit All: Architecture-Aware Adaptive Batch Scheduling with DEBA
por: Belias, François, et al.
Publicado: (2025)

Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?
por: Sommer, Emanuel, et al.
Publicado: (2026)