Cover Image

Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Di, Wu, Xun, Huang, Shaohan, Wang, Yudong, Shao, Hanyong, Hao, Yingbo, Chi, Zewen, Dong, Li, Song, Ting, Xia, Yan, Sui, Zhifang, Wei, Furu
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

BitNet Distillation
by: Wu, Xun, et al.
Published: (2025)

BitNet b1.58 2B4T Technical Report
by: Ma, Shuming, et al.
Published: (2025)

BitNet a4.8: 4-bit Activations for 1-bit LLMs
by: Wang, Hongyu, et al.
Published: (2024)

1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
by: Wang, Jinheng, et al.
Published: (2024)

BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
by: Wang, Hongyu, et al.
Published: (2025)

SlideSparse: Fast and Flexible (2N-2):2N Structured Sparsity
by: Shao, Hanyong, et al.
Published: (2026)

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
by: Nielsen, Jacob, et al.
Published: (2024)

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
by: Ma, Shuming, et al.
Published: (2024)

Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
by: Nielsen, Jacob, et al.
Published: (2025)

BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks
by: Nielsen, Jacob, et al.
Published: (2024)

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
by: Zhang, Di, et al.
Published: (2025)

Black-Box On-Policy Distillation of Large Language Models
by: Ye, Tianzhu, et al.
Published: (2025)

On-Policy RL with Optimal Reward Baseline
by: Hao, Yaru, et al.
Published: (2025)

1.58-bit FLUX
by: Yang, Chenglin, et al.
Published: (2024)

Reward Reasoning Model
by: Guo, Jiaxin, et al.
Published: (2025)

The Era of Agentic Organization: Learning to Organize with Language Models
by: Chi, Zewen, et al.
Published: (2025)

MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training
by: Kim, Yongwan, et al.
Published: (2026)

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation
by: Wu, Xun, et al.
Published: (2024)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing
by: Kawamura, Masaya, et al.
Published: (2025)

Exploring Activation Patterns of Parameters in Language Models
by: Wang, Yudong, et al.
Published: (2024)

Textual Aesthetics in Large Language Models
by: Jiang, Lingjie, et al.
Published: (2024)

MH-MoE: Multi-Head Mixture-of-Experts
by: Huang, Shaohan, et al.
Published: (2024)

Multi-Head Mixture-of-Experts
by: Wu, Xun, et al.
Published: (2024)

BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference
by: Zhang, Wenlun, et al.
Published: (2025)

Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits
by: Kim, Dongwon, et al.
Published: (2025)

Think Only When You Need with Large Hybrid-Reasoning Models
by: Jiang, Lingjie, et al.
Published: (2025)

Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs
by: Yang, Zhe, et al.
Published: (2024)

Hybrid Gated Flow (HGF): Stabilizing 1.58-bit LLMs via Selective Low-Rank Correction
by: Pizzo, David Alejandro Trejo
Published: (2026)

Towards Better RL Training Data Utilization via Second-Order Rollout
by: Yang, Zhe, et al.
Published: (2026)

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
by: Liu, Hongyi, et al.
Published: (2025)

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
by: Li, Zongqian, et al.
Published: (2026)

On-Policy Context Distillation for Language Models
by: Ye, Tianzhu, et al.
Published: (2026)

ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
by: Yuan, Zhengqing, et al.
Published: (2024)

Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodings
by: Connor, Richard, et al.
Published: (2025)

Optimizing Prompts for Text-to-Image Generation
by: Hao, Yaru, et al.
Published: (2022)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

Adapting Large Language Models to Domains via Reading Comprehension
by: Cheng, Daixuan, et al.
Published: (2023)

BitAbuse: A Dataset of Visually Perturbed Texts for Defending Phishing Attacks
by: Lee, Hanyong, et al.
Published: (2025)

An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
by: Steinmetz, Cody, et al.
Published: (2025)