Cover Image

Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Harma, Simla Burcu, Chakraborty, Ayan, Sperry, Nicholas, Falsafi, Babak, Jaggi, Martin, Oh, Yunho
Format:	Preprint
Published:	2022
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Effective Interplay between Sparsity and Quantization: From Theory to Practice
by: Harma, Simla Burcu, et al.
Published: (2024)

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
by: Hägele, Alexander, et al.
Published: (2024)

MAx-DNN: Multi-Level Arithmetic Approximation for Energy-Efficient DNN Hardware Accelerators
by: Leon, Vasileios, et al.
Published: (2025)

Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training
by: Kosson, Atli, et al.
Published: (2024)

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training
by: Yoon, Daegun, et al.
Published: (2023)

Towards Fully FP8 GEMM LLM Training at Scale
by: Hernández-Cano, Alejandro, et al.
Published: (2025)

4-bit Shampoo for Memory-Efficient Network Training
by: Wang, Sike, et al.
Published: (2024)

'1'-bit Count-based Sorting Unit to Reduce Link Power in DNN Accelerators
by: Han, Ruichi, et al.
Published: (2026)

Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler
by: Dremov, Aleksandr, et al.
Published: (2025)

Iterative Assessment and Improvement of DNN Operational Accuracy
by: Guerriero, Antonio, et al.
Published: (2023)

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
by: Chmiel, Brian, et al.
Published: (2021)

DNN Modularization via Activation-Driven Training
by: Ngo, Tuan, et al.
Published: (2024)

Empirical Capacity Model for Self-Attention Neural Networks
by: Härmä, Aki, et al.
Published: (2024)

Stochastic Difference-of-Convex Optimization with Momentum
by: Chayti, El Mahdi, et al.
Published: (2025)

A Split-Client Approach to Second-Order Optimization
by: Chayti, El Mahdi, et al.
Published: (2025)

A New First-Order Meta-Learning Algorithm with Convergence Guarantees
by: Chayti, El Mahdi, et al.
Published: (2024)

On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic
by: Park, Yeachan, et al.
Published: (2024)

Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifier
by: Jin, Gaojie, et al.
Published: (2025)

Rethinking the Potential of Layer Freezing for Efficient DNN Training
by: Yang, Chence, et al.
Published: (2025)

Transfer Learning for Temporal Link Prediction
by: Chatterjee, Ayan, et al.
Published: (2025)

HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation
by: Zhou, Xinyu, et al.
Published: (2024)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
by: Kosson, Atli, et al.
Published: (2023)

Benchmarking Optimizers for Large Language Model Pretraining
by: Semenov, Andrei, et al.
Published: (2025)

Deep Grokking: Would Deep Neural Networks Generalize Better?
by: Fan, Simin, et al.
Published: (2024)

KL for a KL: On-Policy Distillation with Control Variate Baseline
by: Oh, Minjae, et al.
Published: (2026)

Float8@2bits: Entropy Coding Enables Data-Free Model Compression
by: Putzky, Patrick, et al.
Published: (2026)

Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance
by: Piao, XinYu, et al.
Published: (2021)

BitNet a4.8: 4-bit Activations for 1-bit LLMs
by: Wang, Hongyu, et al.
Published: (2024)

MLTCP: Congestion Control for DNN Training
by: Rajasekaran, Sudarsanan, et al.
Published: (2024)

How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines
by: Sengupta, Ayan, et al.
Published: (2025)

Exploring Variance Reduction in Importance Sampling for Efficient DNN Training
by: Kutsuna, Takuro
Published: (2025)

DecTrain: Deciding When to Train a Monocular Depth DNN Online
by: Fu, Zih-Sing, et al.
Published: (2024)

Personalized Collaborative Fine-Tuning for On-Device Large Language Models
by: Wagner, Nicolas, et al.
Published: (2024)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection
by: Messmer, Bettina, et al.
Published: (2025)

Gradient-Normalized Smoothness for Optimization with Approximate Hessians
by: Semenov, Andrei, et al.
Published: (2025)

CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
by: Mohtashami, Amirkeivan, et al.
Published: (2023)

GRAPE: Optimize Data Mixture for Group Robust Multi-target Adaptive Pretraining
by: Fan, Simin, et al.
Published: (2025)

Fixed points of nonnegative neural networks
by: Piotrowski, Tomasz J., et al.
Published: (2021)

EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations
by: Choi, Sangun, et al.
Published: (2025)

Revisiting DNN Training for Intermittently-Powered Energy-Harvesting Micro-Computers
by: Mishra, Cyan Subhra, et al.
Published: (2024)