Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910046990893056 |
|---|---|
| author | Chhugani, Jatin Jeong, Geonhwa Su, Bor-Yiing Pan, Yunjie Yang, Hanmei Ankit, Aayush Yu, Jiecao Deng, Summer Chen, Yunqing Satish, Nadathur Kim, Changkyu |
| author_facet | Chhugani, Jatin Jeong, Geonhwa Su, Bor-Yiing Pan, Yunjie Yang, Hanmei Ankit, Aayush Yu, Jiecao Deng, Summer Chen, Yunqing Satish, Nadathur Kim, Changkyu |
| contents | Large Language Models (LLMs) have intensified the need for low-precision formats that enable efficient, large-scale inference. The Open Compute Project (OCP) Microscaling (MX) standard is attractive due to its favorable hardware efficiency, but its 4-bit variant (MXFP4) lags behind NVIDIA's NVFP4 in accuracy, limiting adoption. We introduce two software-only techniques, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS), that improve MXFP4 quantization fidelity without requiring hardware changes. OAS reduces overall errors by increasing effective dynamic range under power-of-two block scaling, while MBS allocates higher-precision scaling at a coarser granularity to better preserve outliers. Across multiple LLMs and standard downstream benchmarks, OAS and MBS reduce the end-to-end accuracy gap between MXFP4 and NVFP4 from about 10% to below 1% on average, while incurring modest GEMM overhead (6.2% on average). These results re-establish MXFP4 as a practical alternative to NVFP4, enabling near-NVFP4 accuracy while retaining MX's hardware-efficiency advantages (e.g., 12% relative area savings in tensor cores). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08713 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction Chhugani, Jatin Jeong, Geonhwa Su, Bor-Yiing Pan, Yunjie Yang, Hanmei Ankit, Aayush Yu, Jiecao Deng, Summer Chen, Yunqing Satish, Nadathur Kim, Changkyu Hardware Architecture Artificial Intelligence Machine Learning Performance Large Language Models (LLMs) have intensified the need for low-precision formats that enable efficient, large-scale inference. The Open Compute Project (OCP) Microscaling (MX) standard is attractive due to its favorable hardware efficiency, but its 4-bit variant (MXFP4) lags behind NVIDIA's NVFP4 in accuracy, limiting adoption. We introduce two software-only techniques, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS), that improve MXFP4 quantization fidelity without requiring hardware changes. OAS reduces overall errors by increasing effective dynamic range under power-of-two block scaling, while MBS allocates higher-precision scaling at a coarser granularity to better preserve outliers. Across multiple LLMs and standard downstream benchmarks, OAS and MBS reduce the end-to-end accuracy gap between MXFP4 and NVFP4 from about 10% to below 1% on average, while incurring modest GEMM overhead (6.2% on average). These results re-establish MXFP4 as a practical alternative to NVFP4, enabling near-NVFP4 accuracy while retaining MX's hardware-efficiency advantages (e.g., 12% relative area savings in tensor cores). |
| title | Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction |
| topic | Hardware Architecture Artificial Intelligence Machine Learning Performance |
| url | https://arxiv.org/abs/2603.08713 |