Image de couverture de livre

Training LLMs with MXFP4

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tseng, Albert, Yu, Tao, Park, Youngsuk
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Stochastic Rounding for LLM Training: Theory and Practice
par: Ozkara, Kaan, et autres
Publié: (2025)

Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
par: Bian, Song, et autres
Publié: (2025)

Oscillation-Reduced MXFP4 Training for Vision Transformers
par: Chen, Yuxiang, et autres
Publié: (2025)

Recipes for Pre-training LLMs with MXFP8
par: Mishra, Asit, et autres
Publié: (2025)

TritonRL: Training LLMs to Think and Code Triton Without Cheating
par: Woo, Jiin, et autres
Publié: (2025)

Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs
par: Liu, Hongyi, et autres
Publié: (2025)

Block Rotation is All You Need for MXFP4 Quantization
par: Shao, Yuantian, et autres
Publié: (2025)

TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
par: Xu, Zukang, et autres
Publié: (2026)

Pretraining large language models with MXFP4 on Native FP4 Hardware
par: Cim, Musa, et autres
Publié: (2026)

MuonBP: Faster Muon via Block-Periodic Orthogonalization
par: Khaled, Ahmed, et autres
Publié: (2025)

Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
par: Chhugani, Jatin, et autres
Publié: (2026)

Collage: Light-Weight Low-Precision Strategy for LLM Training
par: Yu, Tao, et autres
Publié: (2024)

Diagonal-Tiled Mixed-Precision Attention for Efficient Low-Bit MXFP Inference
par: Ding, Yifu, et autres
Publié: (2026)

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
par: Liu, Hongyi, et autres
Publié: (2025)

MXNorm: Reusing MXFP block scales for efficient tensor normalisation
par: McLean, Callum, et autres
Publié: (2026)

Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
par: Li, Xiaocan, et autres
Publié: (2026)

Online Posterior Sampling with a Diffusion Prior
par: Kveton, Branislav, et autres
Publié: (2024)

Shadow Cones: A Generalized Framework for Partial Order Embeddings
par: Yu, Tao, et autres
Publié: (2023)

Theoretical Guarantees of Learning Ensembling Strategies with Applications to Time Series Forecasting
par: Hasson, Hilaf, et autres
Publié: (2023)

L$^3$: Large Lookup Layers
par: Tseng, Albert, et autres
Publié: (2026)

Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
par: Gautam, Tanmay, et autres
Publié: (2024)

Metis: Training LLMs with FP4 Quantization
par: Cao, Hengjie, et autres
Publié: (2025)

Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
par: Deng, Wenlong, et autres
Publié: (2026)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
par: Ouyang, Xu, et autres
Publié: (2024)

Model-Preserving Adaptive Rounding
par: Tseng, Albert, et autres
Publié: (2025)

MuCon: Clipped Muon Updates for LLM Training
par: Yi, Albert
Publié: (2026)

QTIP: Quantization with Trellises and Incoherence Processing
par: Tseng, Albert, et autres
Publié: (2024)

RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
par: Wei, Quan, et autres
Publié: (2025)

Verifier-free Test-Time Sampling for Vision Language Action Models
par: Jang, Suhyeok, et autres
Publié: (2025)

Training Dynamics Impact Post-Training Quantization Robustness
par: Catalan-Tatjer, Albert, et autres
Publié: (2025)

MedM2T: A MultiModal Framework for Time-Aware Modeling with Electronic Health Record and Electrocardiogram Data
par: Kuo, Yu-Chen, et autres
Publié: (2025)

FP4 All the Way: Fully Quantized Training of LLMs
par: Chmiel, Brian, et autres
Publié: (2025)

Laplace Approximation For Tensor Train Kernel Machines In System Identification
par: Saiapin, Albert, et autres
Publié: (2025)

Physics-Informed Neural Network for Predicting Out-of-Training-Range TCAD Solution with Minimized Domain Expertise
par: Lu, Albert, et autres
Publié: (2024)

StreetMath: Study of LLMs' Approximation Behaviors
par: Tseng, Chiung-Yi, et autres
Publié: (2025)

Learning-Based WiFi Fingerprint Inpainting via Generative Adversarial Networks
par: Chan, Yu, et autres
Publié: (2024)

LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters
par: Chang, Ching, et autres
Publié: (2023)

CAAP: Class-Dependent Automatic Data Augmentation Based On Adaptive Policies For Time Series
par: Chang, Tien-Yu, et autres
Publié: (2024)

Efficient RL Training for LLMs with Experience Replay
par: Arnal, Charles, et autres
Publié: (2026)

Heterogeneous Low-Bandwidth Pre-Training of LLMs
par: Obeidi, Yazan, et autres
Publié: (2026)