Copertina

HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Jinhao Zhang Yunquan, yan, Zicheng, Zhang, Boyang, Sun, Jun, Cheng, Daning
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization
di: Zhang, Jinhao, et al.
Pubblicazione: (2025)

MoQE: Improve Quantization Model performance via Mixture of Quantization Experts
di: Zhang, Jinhao, et al.
Pubblicazione: (2025)

FP=xINT:Representing Neural Networks via Low-Bit Series Basis Functions
di: Zhang, Boyang, et al.
Pubblicazione: (2024)

Compression for Better: A General and Stable Lossless Compression Framework
di: Zhang, Boyang, et al.
Pubblicazione: (2024)

MoE-DisCo:Low Economy Cost Training Mixture-of-Experts Models
di: Ye, Xin, et al.
Pubblicazione: (2026)

Lossless Model Compression via Joint Low-Rank Factorization Optimization
di: Zhang, Boyang, et al.
Pubblicazione: (2024)

A General Error-Theoretical Analysis Framework for Constructing Compression Strategies
di: Zhang, Boyang, et al.
Pubblicazione: (2025)

A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
di: Cheng, Daning, et al.
Pubblicazione: (2026)

Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training
di: Liu, Zeyu, et al.
Pubblicazione: (2025)

Rethinking Parameter Sharing as Graph Coloring for Structured Compression
di: Zhang, Boyang, et al.
Pubblicazione: (2025)

HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs
di: Wang, Guoan, et al.
Pubblicazione: (2026)

Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
di: Zhong, Yi, et al.
Pubblicazione: (2026)

CaRoBio: 3D Cable Routing with a Bio-inspired Gripper Fingernail
di: Zuo, Jiahui, et al.
Pubblicazione: (2025)

Q$^2$: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantization
di: Wang, Zhaoyang, et al.
Pubblicazione: (2025)

HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC
di: Li, Maoliang, et al.
Pubblicazione: (2026)

LittleBit: Ultra Low-Bit Quantization via Latent Factorization
di: Lee, Banseok, et al.
Pubblicazione: (2025)

Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression
di: Zhang, Xi, et al.
Pubblicazione: (2025)

Why Do Some Inputs Break Low-Bit LLM Quantization?
di: Chang, Ting-Yun, et al.
Pubblicazione: (2025)

LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs
di: Lee, Jung Hyun, et al.
Pubblicazione: (2026)

Efficient Edge LLMs Deployment via HessianAware Quantization and CPU GPU Collaborative
di: Zhang, Tuo, et al.
Pubblicazione: (2025)

Verification of Bit-Flip Attacks against Quantized Neural Networks
di: Zhang, Yedi, et al.
Pubblicazione: (2025)

Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
di: Lee, Deokjae, et al.
Pubblicazione: (2025)

SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs
di: Cheng, Wenhua, et al.
Pubblicazione: (2025)

Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
di: Zhang, Tianao, et al.
Pubblicazione: (2025)

Can the capability of Large Language Models be described by human ability? A Meta Study
di: Zan, Mingrui, et al.
Pubblicazione: (2025)

Attn-QAT: 4-Bit Attention With Quantization-Aware Training
di: Zhang, Peiyuan, et al.
Pubblicazione: (2026)

MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity
di: Choi, Kanghyun, et al.
Pubblicazione: (2024)

PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models
di: Zhao, Jiaqi, et al.
Pubblicazione: (2025)

Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
di: Li, Zhen, et al.
Pubblicazione: (2025)

EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection
di: Zhou, Yuhao, et al.
Pubblicazione: (2025)

Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models
di: Rakotoarivony, Lucas
Pubblicazione: (2026)

R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
di: Chen, Jiayi, et al.
Pubblicazione: (2025)

Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
di: Wu, Haoning, et al.
Pubblicazione: (2023)

Q-ROAR: Outlier-Aware Rescaling for RoPE Position Interpolation in Quantized Long-Context LLMs
di: Qiao, Ye, et al.
Pubblicazione: (2025)

InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
di: Li, Ke, et al.
Pubblicazione: (2026)

SplitQuant: Layer Splitting for Low-Bit Neural Network Quantization
di: Song, Jaewoo, et al.
Pubblicazione: (2025)

Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision
di: Huang, Xijie, et al.
Pubblicazione: (2023)

RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache
di: Zhang, Junkai, et al.
Pubblicazione: (2026)

ECQ$^{\text{x}}$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs
di: Becking, Daniel, et al.
Pubblicazione: (2021)

D$^2$Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
di: Yan, Xianglong, et al.
Pubblicazione: (2026)