Cover Image

BitSnap: Checkpoint Sparsification and Quantization in LLM Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peng, Yanxin, Li, Qingping, Wu, Baodong, Li, Shigang, Dai, Guohao, Yan, Shengen, Wang, Yu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MSQ: Memory-Efficient Bit Sparsification Quantization
by: Han, Seokho, et al.
Published: (2025)

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
by: Wang, Luning, et al.
Published: (2024)

One-Bit Quantization and Sparsification for Multiclass Linear Classification with Strong Regularization
by: Ghane, Reza, et al.
Published: (2024)

Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification
by: Huang, Hong, et al.
Published: (2026)

TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction
by: Li, Yuhang, et al.
Published: (2024)

Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models with Conditional Score Distillation
by: Liu, Enshu, et al.
Published: (2025)

Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
by: Zhang, Tianao, et al.
Published: (2025)

Efficient Distributed Training through Gradient Compression with Sparsification and Quantization Techniques
by: Singh, Shruti, et al.
Published: (2024)

SBVR: Summation of BitVector Representation for Efficient LLM Quantization
by: Bang, Wonjun, et al.
Published: (2025)

HBVLA: Pushing 1-Bit Post-Training Quantization for Vision-Language-Action Models
by: Yan, Xin, et al.
Published: (2026)

Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression
by: Zhang, Xi, et al.
Published: (2025)

Inshrinkerator: Compressing Deep Learning Training Checkpoints via Dynamic Quantization
by: Agrawal, Amey, et al.
Published: (2023)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
by: Liu, Enshu, et al.
Published: (2024)

InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
by: Li, Ke, et al.
Published: (2026)

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs
by: Wang, Haoyu, et al.
Published: (2024)

Bits for Privacy: Evaluating Post-Training Quantization via Membership Inference
by: Zhang, Chenxiang, et al.
Published: (2025)

QQQ: Quality Quattuor-Bit Quantization for Large Language Models
by: Zhang, Ying, et al.
Published: (2024)

DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference
by: Zhou, Jiajun, et al.
Published: (2023)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
by: Ouyang, Xu, et al.
Published: (2024)

Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
by: Wu, Junyi, et al.
Published: (2026)

Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths
by: Fu, Tianyu, et al.
Published: (2024)

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
by: Zeng, Chao, et al.
Published: (2024)

BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
by: Chen, Junyu, et al.
Published: (2026)

SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
by: Jia, Jinda, et al.
Published: (2026)

STAlloc: Enhancing Memory Efficiency in Large-Scale Model Training with Spatio-Temporal Planning
by: Huang, Zixiao, et al.
Published: (2025)

From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
by: Deng, Kaiyuan, et al.
Published: (2026)

AdaQAT: Adaptive Bit-Width Quantization-Aware Training
by: Gernigon, Cédric, et al.
Published: (2024)

Attn-QAT: 4-Bit Attention With Quantization-Aware Training
by: Zhang, Peiyuan, et al.
Published: (2026)

Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
by: Lee, Deokjae, et al.
Published: (2025)

SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
by: Jia, Jinda, et al.
Published: (2024)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
by: Zhang, Chao, et al.
Published: (2025)

DecDEC: A Systems Approach to Advancing Low-Bit LLM Quantization
by: Park, Yeonhong, et al.
Published: (2024)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
by: Zhao, Jiayu, et al.
Published: (2026)

E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling
by: Yuan, Zhihang, et al.
Published: (2024)

DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training
by: Wang, Yuanqing, et al.
Published: (2026)

Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
by: Wang, YiFeng, et al.
Published: (2026)

FrameQuant: Flexible Low-Bit Quantization for Transformers
by: Adepu, Harshavardhan, et al.
Published: (2024)

Why Do Some Inputs Break Low-Bit LLM Quantization?
by: Chang, Ting-Yun, et al.
Published: (2025)

BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration
by: Chen, Yuzong, et al.
Published: (2024)

COPF: An Online Framework for Deployment-Stable Counterfactual Fairness in Evolving Graphs
by: Li, Sheng'en, et al.
Published: (2026)