Cover Image

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nielsen, Jacob, Galke, Lukas, Schneider-Kamp, Peter
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
by: Nielsen, Jacob, et al.
Published: (2025)

BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks
by: Nielsen, Jacob, et al.
Published: (2024)

BitNet b1.58 2B4T Technical Report
by: Ma, Shuming, et al.
Published: (2025)

BitNet a4.8: 4-bit Activations for 1-bit LLMs
by: Wang, Hongyu, et al.
Published: (2024)

BitNet Distillation
by: Wu, Xun, et al.
Published: (2025)

BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
by: Wang, Hongyu, et al.
Published: (2025)

Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
by: Zhang, Di, et al.
Published: (2026)

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
by: Ma, Shuming, et al.
Published: (2024)

1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
by: Wang, Jinheng, et al.
Published: (2024)

SDUs DAISY: A Benchmark for Danish Culture
by: Nielsen, Jacob, et al.
Published: (2026)

An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
by: Steinmetz, Cody, et al.
Published: (2025)

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes
by: From, Mogens Henrik, et al.
Published: (2025)

1.58-bit FLUX
by: Yang, Chenglin, et al.
Published: (2024)

FlexMoRE: A Flexible Mixture of Rank-heterogeneous Experts for Efficient Federatedly-trained Large Language Models
by: Pirchert, Annemette Brok, et al.
Published: (2026)

Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
by: Torrielli, Federico, et al.
Published: (2026)

BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing
by: Kawamura, Masaya, et al.
Published: (2025)

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks
by: Nielsen, Dan Saattrup, et al.
Published: (2024)

FrameQuant: Flexible Low-Bit Quantization for Transformers
by: Adepu, Harshavardhan, et al.
Published: (2024)

Are LLM-based methods good enough for detecting unfair terms of service?
by: Frasheri, Mirgita, et al.
Published: (2024)

LittleBit: Ultra Low-Bit Quantization via Latent Factorization
by: Lee, Banseok, et al.
Published: (2025)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
by: Zhang, Chao, et al.
Published: (2025)

AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
by: IslamBouli, Beshr, et al.
Published: (2026)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by: Ouyang, Xu, et al.
Published: (2024)

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
by: Liu, Zirui, et al.
Published: (2024)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
by: Liao, Baohao, et al.
Published: (2024)

QuIP: 2-Bit Quantization of Large Language Models With Guarantees
by: Chee, Jerry, et al.
Published: (2023)

QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
by: Zandieh, Amir, et al.
Published: (2024)

1-Bit FQT: Pushing the Limit of Fully Quantized Training to 1-bit
by: Gao, Chang, et al.
Published: (2024)

Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
by: Wang, YiFeng, et al.
Published: (2026)

MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training
by: Kim, Yongwan, et al.
Published: (2026)

Training Language Models to Use Prolog as a Tool
by: Mellgren, Niklas, et al.
Published: (2025)

Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations
by: Blumenberg, Patrick, et al.
Published: (2025)

DaLA: Danish Linguistic Acceptability Evaluation Guided by Real World Errors
by: Barmina, Gianluca, et al.
Published: (2025)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
by: Song, Yurun, et al.
Published: (2025)

pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training
by: Zhang, Wenzheng, et al.
Published: (2026)

More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantization
by: Ichikawa, Yuma, et al.
Published: (2025)

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
by: Xu, Bingxin, et al.
Published: (2025)

Hybrid Gated Flow (HGF): Stabilizing 1.58-bit LLMs via Selective Low-Rank Correction
by: Pizzo, David Alejandro Trejo
Published: (2026)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
by: Park, Jungwoo, et al.
Published: (2025)

MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
by: Wang, Dongwei, et al.
Published: (2026)