Cover Image

The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cao, Hengjie, Huang, Zhendong, Chen, Mengyi, Yang, Yifeng, Yu, Fanqi, Huang, Ruijun, Dong, Fang, Zhang, Xin, Zhou, Jixian, Chen, Anrui, Dong, Mingzhi, Wang, Yujiang, Hou, Jinlong, Lv, Qin, Cheng, Yuan, Lu, Tun, Yang, Fan, Shang, Li
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Metis: Training LLMs with FP4 Quantization
by: Cao, Hengjie, et al.
Published: (2025)

Dispelling the Curse of Singularities in Neural Network Optimizations
by: Cao, Hengjie, et al.
Published: (2026)

Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers
by: Chen, Anrui, et al.
Published: (2026)

Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy
by: Huang, Zhendong, et al.
Published: (2026)

SD-MoE: Spectral Decomposition for Effective Expert Specialization
by: Huang, Ruijun, et al.
Published: (2026)

The Blessing and Curse of Dimensionality in Safety Alignment
by: Teo, Rachel S. Y., et al.
Published: (2025)

U.S. GPO CD-ROMs: Blessing or Curse?
by: Stubbs, Walter, et al.
Published: (1999)

Monotone Missing Data: A Blessing and a Curse
by: Tikka, Santtu, et al.
Published: (2024)

LoS Blockage in Pinching-Antenna Systems: Curse or Blessing?
by: Ding, Zhiguo, et al.
Published: (2025)

Smartphone Usage in Academia: A Blessing or A Curse?
by: Farhad Mohammad Afzali
Published: (2022)

Information Disclosure in Preemption Races: Blessing or (Winner's) Curse?
by: Catherine Bobtcheff, et al.
Published: (2025)

Differentiated Relational Strategies in Major Supplier Networks: A Blessing or a Curse in Collectivist Cultures?
by: Zebin Yan, et al.
Published: (2025)

Denoising Reuse: Exploiting Inter-frame Motion Consistency for Efficient Video Latent Generation
by: Wang, Chenyu, et al.
Published: (2024)

Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models
by: Shi, Yubin, et al.
Published: (2024)

Project Archetypes: A Blessing and a Curse for AI Development
by: Dolata, Mateusz, et al.
Published: (2024)

LLM-FP4: 4-Bit Floating-Point Quantized Transformers
by: Liu, Shih-yang, et al.
Published: (2023)

FuseChat: Knowledge Fusion of Chat Models
by: Wan, Fanqi, et al.
Published: (2024)

Neural Quantum States and Peaked Molecular Wave Functions: Curse or Blessing?
by: Malyshev, Aleksei, et al.
Published: (2024)

Artificial Intelligence in Climate and Sustainable Finance: A Blessing or a Curse?
by: Filippo di Pietro, et al.
Published: (2026)

Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibria
by: Liang, Tengyuan
Published: (2022)

AI Diffusion to Low- and Middle Income Countries; A Blessing or a Curse?
by: Lipcsey, Rafael Andersson
Published: (2024)

Welfare Resource Curse and Blessing? Accounting for Causal Heterogeneity in the Oil‐Welfare Nexus
by: Pablo Garcés‐Velástegui
Published: (2026)

GPTVQ: The Blessing of Dimensionality for LLM Quantization
by: van Baalen, Mart, et al.
Published: (2024)

The Conundrum of Diffuse Basis Sets: A Blessing for Accuracy yet a Curse for Sparsity
by: Laqua, Henryk, et al.
Published: (2025)

Curse or Blessing: Investigating the Influence of Firms’ Artificial Intelligence Adoption on Employee Job Satisfaction
by: Colin Schulz, et al.
Published: (2025)

The IP3R2 Knockout Mice in Behavior: A Blessing or a Curse?
by: Joana Gonçalves‐Ribeiro, et al.
Published: (2025)

Mixed Blessing: Class-Wise Embedding guided Instance-Dependent Partial Label Learning
by: Yang, Fuchao, et al.
Published: (2024)

SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models
by: Yang, Ziyi, et al.
Published: (2025)

Self-Evolution Fine-Tuning for Policy Optimization
by: Chen, Ruijun, et al.
Published: (2024)

BlockPruner: Fine-grained Pruning for Large Language Models
by: Zhong, Longguang, et al.
Published: (2024)

Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
by: Shao, Jiandong, et al.
Published: (2025)

Does Country Risk Change the Financial Resource Curse Into a Blessing? Evidence From an Emerging Economy
by: Faisal Faisal, et al.
Published: (2025)

Medical records condensation: a roadmap towards healthcare data democratisation
by: Wang, Yujiang, et al.
Published: (2023)

Computation-Efficient Backscatter-Blessed MEC with User Reciprocity
by: Gu, Bowen, et al.
Published: (2023)

Global Value Chain Participation and Sustainable Development in Belt and Road Initiative Economies: A Blessing or a Curse?
by: Muhammad Uzair Ali, et al.
Published: (2025)

FP8 Quantization: The Power of the Exponent
by: Kuzmin, Andrey, et al.
Published: (2022)

A Bayesian Bootstrap Approach for Dynamic Borrowing for Minimizing Mean Squared Error
by: Wang, Jixian, et al.
Published: (2024)

Climate‐Dependent Aging of Photovoltaic PET Backsheets: Insights From Arid‐Cold Exposure
by: Yifan Chen, et al.
Published: (2026)

Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning
by: Zhao, Maosen, et al.
Published: (2025)

Enabling and Enacting: Interplay of Agency and Emotions in the Learning Experiences of Successful EFL Learners
by: Hengjie Chen, et al.
Published: (2025)