Cover Image

TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Yuxiang, Liu, Yifan, Xu, Xiaoming, Zhang, Pengle, Beyer, Michael, Rapp, Martin, Zhu, Jun, Chen, Jianfei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Oscillation-Reduced MXFP4 Training for Vision Transformers
by: Chen, Yuxiang, et al.
Published: (2025)

Accurate INT8 Training Through Dynamic Block-Level Fallback
by: Zhang, Pengle, et al.
Published: (2025)

SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
by: Zhang, Jintao, et al.
Published: (2024)

Deterministic Differentiable Structured Pruning for Large Language Models
by: Huang, Weiyu, et al.
Published: (2026)

Dissecting Outlier Dynamics in LLM NVFP4 Pretraining
by: Dong, Peijie, et al.
Published: (2026)

SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
by: Zhang, Jintao, et al.
Published: (2024)

SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
by: Zhang, Jintao, et al.
Published: (2025)

Pretraining Large Language Models with NVFP4
by: NVIDIA, et al.
Published: (2025)

SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization
by: Bao, Chengzhu, et al.
Published: (2026)

Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
by: Panferov, Andrei, et al.
Published: (2026)

Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization
by: Xi, Haocheng, et al.
Published: (2024)

SageAttention2++: A More Efficient Implementation of SageAttention2
by: Zhang, Jintao, et al.
Published: (2025)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
by: Huang, Weiyu, et al.
Published: (2025)

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training
by: Huang, Weiyu, et al.
Published: (2024)

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
by: Cook, Jack, et al.
Published: (2025)

FAAR: Format-Aware Adaptive Rounding for NVFP4
by: Li, Hanglin, et al.
Published: (2026)

MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations
by: Zou, Jiaxiang, et al.
Published: (2026)

Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery
by: Xin, Meng, et al.
Published: (2026)

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
by: Zhang, Jintao, et al.
Published: (2025)

Efficient Backpropagation with Variance-Controlled Adaptive Sampling
by: Wang, Ziteng, et al.
Published: (2024)

S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training
by: Hu, Yuezhou, et al.
Published: (2024)

LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
by: Chen, Yukang, et al.
Published: (2026)

RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
by: Chen, Yuzong, et al.
Published: (2025)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
by: Park, Jungwoo, et al.
Published: (2025)

ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
by: Meng, Haoqian, et al.
Published: (2026)

C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
by: Luo, Tianjiao, et al.
Published: (2024)

Accurate Block Quantization in LLMs with Outliers
by: Trukhanov, Nikita, et al.
Published: (2024)

QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
by: Liu, Jing, et al.
Published: (2023)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
by: Wang, Ziteng, et al.
Published: (2024)

Cross-Image Contrastive Decoding: Precise, Lossless Suppression of Language Priors in Large Vision-Language Models
by: Zhao, Jianfei, et al.
Published: (2025)

Systematic Outliers in Large Language Models
by: An, Yongqi, et al.
Published: (2025)

Image Outlier Detection Without Training using RANSAC
by: Tsai, Chen-Han, et al.
Published: (2023)

Low‐Frequency Oscillation Suppression of Pulse Train Controlled Cuk Converter Operating in Continuous Conduction Mode
by: Yanyan Jin, et al.
Published: (2025)

AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation
by: Kim, Seonggon, et al.
Published: (2026)

Transformer-based Approach for Accurate Asteroid Spectra taxonomy and albedo estimation
by: Tang, Yijun, et al.
Published: (2025)

Tetra(4‐hexylphenyl) [3]‐ and [5]Cumulenes
by: Bozheng Sun, et al.
Published: (2025)

Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization
by: Xiao, Jinying, et al.
Published: (2025)

FASP: Fast and Accurate Structured Pruning of Large Language Models
by: Hu, Hanyu, et al.
Published: (2025)

Phase Noise Suppression and Frequency Stabilization Technology Based on Single‐Fiber‐Loop Optoelectronic Oscillator
by: Yingbo Liu, et al.
Published: (2025)