Cover Image

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Chao, Liu, Songwei, Xie, Yusheng, Liu, Hong, Wang, Xiaojian, Wei, Miao, Yang, Shu, Chen, Fangmin, Mei, Xing
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference
by: Zeng, Chao, et al.
Published: (2024)

Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion
by: Liu, Songwei, et al.
Published: (2025)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
by: Liu, Songwei, et al.
Published: (2024)

Hybrid SD: Edge-Cloud Collaborative Inference for Stable Diffusion Models
by: Yan, Chenqian, et al.
Published: (2024)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
by: Hu, Xing, et al.
Published: (2024)

QQQ: Quality Quattuor-Bit Quantization for Large Language Models
by: Zhang, Ying, et al.
Published: (2024)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
by: Zhang, Chao, et al.
Published: (2025)

Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models
by: Liu, Wanlong, et al.
Published: (2025)

VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
by: Yao, Dingyu, et al.
Published: (2025)

Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
by: Li, Zhen, et al.
Published: (2025)

MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
by: Wang, Huizheng, et al.
Published: (2025)

MBQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantization
by: Zhong, Yunshan, et al.
Published: (2023)

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
by: Zhang, Xinyuan, et al.
Published: (2024)

S2O: Early Stopping for Sparse Attention via Online Permutation
by: Zhang, Yu, et al.
Published: (2026)

KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
by: Zhang, Tianyi, et al.
Published: (2024)

Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores
by: Ma, Shaobo, et al.
Published: (2024)

ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion
by: Peng, Xurui, et al.
Published: (2025)

Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
by: Zhang, Tianao, et al.
Published: (2025)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
by: Liao, Baohao, et al.
Published: (2024)

QuIP: 2-Bit Quantization of Large Language Models With Guarantees
by: Chee, Jerry, et al.
Published: (2023)

AnTKV: Anchor Token-Aware Sub-Bit Vector Quantization for KV Cache in Large Language Models
by: Li, Zeyu, et al.
Published: (2025)

MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization
by: Yu, JiangYong, et al.
Published: (2025)

TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly
by: Koike-Akino, Toshiaki, et al.
Published: (2026)

LLM-FP4: 4-Bit Floating-Point Quantized Transformers
by: Liu, Shih-yang, et al.
Published: (2023)

BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
by: Chen, Junyu, et al.
Published: (2026)

Binary Weight Multi-Bit Activation Quantization for Compute-in-Memory CNN Accelerators
by: Zhou, Wenyong, et al.
Published: (2025)

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
by: Chitty-Venkata, Krishna Teja, et al.
Published: (2024)

Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Models
by: Tao, Keda, et al.
Published: (2025)

NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models
by: Chong, Hyochan, et al.
Published: (2026)

LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
by: Bhatnagar, Shubhang, et al.
Published: (2025)

Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
by: Zhong, Yi, et al.
Published: (2026)

BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models
by: Yan, Xiaobei, et al.
Published: (2025)

FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AI
by: Tahmasebi, Faraz, et al.
Published: (2024)

SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
by: Huang, Wei, et al.
Published: (2024)

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
by: Zhao, Jiayu, et al.
Published: (2026)

BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration
by: Chen, Yuzong, et al.
Published: (2024)

eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization
by: Agrawal, Aditya, et al.
Published: (2024)

Enabling Dynamic Sparsity in Quantized LLM Inference
by: Wang, Rongxiang, et al.
Published: (2025)

WaferLLM: Large Language Model Inference at Wafer Scale
by: He, Congjie, et al.
Published: (2025)

APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
by: Ma, Shaobo, et al.
Published: (2025)