Imagen de Portada

From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Deng, Kaiyuan, Zheng, Hangyu, Qing, Minghai, Zhu, Kunxiong, Li, Gen, Xiao, Yang, Zhang, Lan Emily, Guo, Linke, Hui, Bo, Wang, Yanzhi, Yuan, Geng, Agrawal, Gagan, Niu, Wei, Ma, Xiaolong
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Masking
por: Deng, Kaiyuan, et al.
Publicado: (2026)

FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations
por: Shu, Zhihao, et al.
Publicado: (2026)

Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization
por: Li, Gen, et al.
Publicado: (2025)

LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation
por: Xiao, Yang, et al.
Publicado: (2025)

ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training
por: Ji, Jie, et al.
Publicado: (2026)

Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models
por: Deng, Kaiyuan, et al.
Publicado: (2026)

On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks
por: Huang, Wei, et al.
Publicado: (2023)

The Uniqueness of LLaMA3-70B Series with Per-Channel Quantization
por: Qin, Minghai
Publicado: (2024)

Data Overfitting for On-Device Super-Resolution with Dynamic Algorithm and Compiler Co-Design
por: Li, Gen, et al.
Publicado: (2024)

Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment
por: Tan, Qitao, et al.
Publicado: (2026)

SoD$^2$: Statically Optimizing Dynamic Deep Neural Network
por: Niu, Wei, et al.
Publicado: (2024)

Fast and Memory-Efficient Video Diffusion Using Streamlined Inference
por: Zhan, Zheng, et al.
Publicado: (2024)

Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks
por: Niu, Muqun, et al.
Publicado: (2024)

Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
por: Lee, Deokjae, et al.
Publicado: (2025)

CUDAHercules: Benchmarking Hardware-Aware Expert-level CUDA Optimization for LLMs
por: Li, Shiyang, et al.
Publicado: (2026)

Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
por: Li, Zhengang, et al.
Publicado: (2024)

FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
por: Xie, Xilong, et al.
Publicado: (2025)

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs
por: Wang, Haoyu, et al.
Publicado: (2024)

Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training
por: Zhang, Yangming, et al.
Publicado: (2026)

Predictability‐Aware Subsequence Modeling for Sequential Recommendation
por: Hangyu Deng, et al.
Publicado: (2024)

eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization
por: Agrawal, Aditya, et al.
Publicado: (2024)

HBVLA: Pushing 1-Bit Post-Training Quantization for Vision-Language-Action Models
por: Yan, Xin, et al.
Publicado: (2026)

QS4D: Quantization‐Aware Training for Efficient Hardware Deployment of Structured State‐Space Sequential Models
por: Sebastian Siegel, et al.
Publicado: (2026)

Efficient Edge LLMs Deployment via HessianAware Quantization and CPU GPU Collaborative
por: Zhang, Tuo, et al.
Publicado: (2025)

Streamlined Transmission: A Semantic-Aware XR Deployment Framework Enhanced by Generative AI
por: Yang, Wanting, et al.
Publicado: (2024)

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
por: Ouyang, Xu, et al.
Publicado: (2024)

What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale
por: Yuan, Xiaoyong, et al.
Publicado: (2025)

HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs
por: Wang, Guoan, et al.
Publicado: (2026)

What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
por: Lv, Keyu, et al.
Publicado: (2026)

End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
por: Tan, Qitao, et al.
Publicado: (2025)

Q$^2$: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantization
por: Wang, Zhaoyang, et al.
Publicado: (2025)

AdaQAT: Adaptive Bit-Width Quantization-Aware Training
por: Gernigon, Cédric, et al.
Publicado: (2024)

Attn-QAT: 4-Bit Attention With Quantization-Aware Training
por: Zhang, Peiyuan, et al.
Publicado: (2026)

JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration
por: Wang, Mingzi, et al.
Publicado: (2025)

Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsification
por: Huang, Hong, et al.
Publicado: (2026)

Laconic: Streamlined Load Balancers for SmartNICs
por: Cui, Tianyi, et al.
Publicado: (2024)

HBLLM: Wavelet-Enhanced High-Fidelity 1-Bit Quantization for LLMs
por: Chen, Ningning, et al.
Publicado: (2025)

CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
por: Han, Insu, et al.
Publicado: (2025)

BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
por: Sajid, Md. Ashiq Ul Islam, et al.
Publicado: (2026)

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
por: Su, Zunhai, et al.
Publicado: (2025)