Copertina

Scaling Law for Language Models Training Considering Batch Size

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Shuai, Xian, Wang, Yiding, Wu, Yimeng, Jiang, Xin, Ren, Xiaozhe
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Scaling Laws for Post Training Quantized Large Language Models
di: Xu, Zifei, et al.
Pubblicazione: (2024)

Exploring Scaling Laws for Local SGD in Large Language Model Training
di: He, Qiaozhi, et al.
Pubblicazione: (2024)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
di: Bergsma, Shane, et al.
Pubblicazione: (2025)

SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
di: Chen, Guoxuan, et al.
Pubblicazione: (2024)

Scaling Laws for Floating Point Quantization Training
di: Sun, Xingwu, et al.
Pubblicazione: (2025)

Scaling Laws for Multilingual Language Models
di: He, Yifei, et al.
Pubblicazione: (2024)

Parallel Scaling Law for Language Models
di: Chen, Mouxiang, et al.
Pubblicazione: (2025)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
di: Zhou, Chenxi, et al.
Pubblicazione: (2025)

Scaling Law for Quantization-Aware Training
di: Chen, Mengzhao, et al.
Pubblicazione: (2025)

Batched Contextual Reinforcement: A Task-Scaling Law for Efficient Reasoning
di: Yang, Bangji, et al.
Pubblicazione: (2026)

P$^2$ Law: Scaling Law for Post-Training After Model Pruning
di: Chen, Xiaodong, et al.
Pubblicazione: (2024)

Prescriptive Scaling Laws for Data Constrained Training
di: Lovelace, Justin, et al.
Pubblicazione: (2026)

Scaling Laws for Discriminative Classification in Large Language Models
di: Wyatte, Dean, et al.
Pubblicazione: (2024)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
di: Liew, Seng Pei, et al.
Pubblicazione: (2025)

Performance Law of Large Language Models
di: Wu, Chuhan, et al.
Pubblicazione: (2024)

Scaling Laws for Downstream Task Performance of Large Language Models
di: Isik, Berivan, et al.
Pubblicazione: (2024)

Can Language Models Discover Scaling Laws?
di: Lin, Haowei, et al.
Pubblicazione: (2025)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
di: Charles, Zachary, et al.
Pubblicazione: (2025)

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
di: Sardana, Nikhil, et al.
Pubblicazione: (2023)

RedPajama: an Open Dataset for Training Large Language Models
di: Weber, Maurice, et al.
Pubblicazione: (2024)

Observational Scaling Laws and the Predictability of Language Model Performance
di: Ruan, Yangjun, et al.
Pubblicazione: (2024)

Relative-Based Scaling Law for Neural Language Models
di: Yue, Baoqing, et al.
Pubblicazione: (2025)

Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing
di: Wu, Chen, et al.
Pubblicazione: (2025)

Uncovering Scaling Laws for Large Language Models via Inverse Problems
di: Verma, Arun, et al.
Pubblicazione: (2025)

Training Language Models with Language Feedback at Scale
di: Scheurer, Jérémy, et al.
Pubblicazione: (2023)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
di: Gu, Jiawei, et al.
Pubblicazione: (2024)

How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
di: Schwethelm, Kristian, et al.
Pubblicazione: (2026)

CarbonScaling: Extending Neural Scaling Laws for Carbon Footprint in Large Language Models
di: Jiang, Lei, et al.
Pubblicazione: (2025)

Scaling Laws for Precision
di: Kumar, Tanishq, et al.
Pubblicazione: (2024)

Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3
di: Yoon, Junsang, et al.
Pubblicazione: (2024)

Efficient Post-Training Pruning of Large Language Models with Statistical Correction
di: Yu, Peiqi, et al.
Pubblicazione: (2026)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
di: Liu, Lei, et al.
Pubblicazione: (2025)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
di: Lin, Haowei, et al.
Pubblicazione: (2024)

Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning
di: Fan, Ziqing, et al.
Pubblicazione: (2025)

Sliding Window Attention Training for Efficient Large Language Models
di: Fu, Zichuan, et al.
Pubblicazione: (2025)

Linear Dynamics in the RLVR Training of Large Language Models
di: Wang, Tianle, et al.
Pubblicazione: (2026)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
di: Chen, Yanxi, et al.
Pubblicazione: (2024)

Scaling Laws For Mixed Quantization
di: Cao, Zeyu, et al.
Pubblicazione: (2024)

Neural Neural Scaling Laws
di: Hu, Michael Y., et al.
Pubblicazione: (2026)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
di: Ouyang, Xu, et al.
Pubblicazione: (2024)