:: Library Catalog

Copertina

Salvato in:

Dettagli Bibliografici
Autori principali:	Hägele, Alexander, Bakouch, Elie, Kosson, Atli, Allal, Loubna Ben, Von Werra, Leandro, Jaggi, Martin
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	https://arxiv.org/abs/2405.18392
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler
di: Dremov, Aleksandr, et al.
Pubblicazione: (2025)

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training
di: Kosson, Atli, et al.
Pubblicazione: (2024)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
di: Kosson, Atli, et al.
Pubblicazione: (2023)

Weight Decay may matter more than muP for Learning Rate Transfer in Practice
di: Kosson, Atli, et al.
Pubblicazione: (2025)

Towards Fully FP8 GEMM LLM Training at Scale
di: Hernández-Cano, Alejandro, et al.
Pubblicazione: (2025)

Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training
di: Harma, Simla Burcu, et al.
Pubblicazione: (2022)

Navigating Scaling Laws: Compute Optimality in Adaptive Model Training
di: Anagnostidis, Sotiris, et al.
Pubblicazione: (2023)

How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
di: Niklaus, Joel, et al.
Pubblicazione: (2026)

The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
di: Schaipp, Fabian, et al.
Pubblicazione: (2025)

On Neural Scaling Laws for Weather Emulation through Continual Training
di: Subramanian, Shashank, et al.
Pubblicazione: (2026)

Scaling Law for Quantization-Aware Training
di: Chen, Mengzhao, et al.
Pubblicazione: (2025)

Scaling Laws for Post Training Quantized Large Language Models
di: Xu, Zifei, et al.
Pubblicazione: (2024)

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
di: Penedo, Guilherme, et al.
Pubblicazione: (2025)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
di: Penedo, Guilherme, et al.
Pubblicazione: (2024)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
di: Qiu, Shikai, et al.
Pubblicazione: (2025)

SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
di: Allal, Loubna Ben, et al.
Pubblicazione: (2025)

Prescriptive Scaling Laws for Data Constrained Training
di: Lovelace, Justin, et al.
Pubblicazione: (2026)

The Sustainability Gap in Robotics: A Large-Scale Survey of Sustainability Awareness in 50,000 Research Articles
di: Skuric, Antun, et al.
Pubblicazione: (2026)

Compute-Optimal Quantization-Aware Training
di: Dremov, Aleksandr, et al.
Pubblicazione: (2025)

Student-Informed Teacher Training
di: Messikommer, Nico, et al.
Pubblicazione: (2024)

Training Transformers for Mesh-Based Simulations
di: Garnier, Paul, et al.
Pubblicazione: (2025)

CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
di: Mohtashami, Amirkeivan, et al.
Pubblicazione: (2023)

Scaling Laws for Floating Point Quantization Training
di: Sun, Xingwu, et al.
Pubblicazione: (2025)

Test-Time Training Scaling Laws for Chemical Exploration in Drug Design
di: Thomas, Morgan, et al.
Pubblicazione: (2025)

Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models
di: Nimmaturi, Datta, et al.
Pubblicazione: (2025)

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
di: Sardana, Nikhil, et al.
Pubblicazione: (2023)

DABstep: Data Agent Benchmark for Multi-step Reasoning
di: Egg, Alex, et al.
Pubblicazione: (2025)

Scaling Law for Language Models Training Considering Batch Size
di: Shuai, Xian, et al.
Pubblicazione: (2024)

Muon in Associative Memory Learning: Training Dynamics and Scaling Laws
di: Li, Binghui, et al.
Pubblicazione: (2026)

SmolVLM: Redefining small and efficient multimodal models
di: Marafioti, Andrés, et al.
Pubblicazione: (2025)

P$^2$ Law: Scaling Law for Post-Training After Model Pruning
di: Chen, Xiaodong, et al.
Pubblicazione: (2024)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
di: Zhang, Yaxiang, et al.
Pubblicazione: (2026)

When Learning Hurts: Fixed-Pole RNN for Real-Time Online Training
di: Morgan, Alexander, et al.
Pubblicazione: (2026)

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
di: Singh, Avi, et al.
Pubblicazione: (2023)

Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design
di: Alabdulmohsin, Ibrahim, et al.
Pubblicazione: (2023)

Exploring Scaling Laws for Local SGD in Large Language Model Training
di: He, Qiaozhi, et al.
Pubblicazione: (2024)

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
di: Chen, Zhengyu, et al.
Pubblicazione: (2025)

Training Compute-Optimal Protein Language Models
di: Cheng, Xingyi, et al.
Pubblicazione: (2024)

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining
di: Fan, Dongyang, et al.
Pubblicazione: (2025)

Scaling Laws for Optimal Data Mixtures
di: Shukor, Mustafa, et al.
Pubblicazione: (2025)