Cover Image

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Park, Jungwoo, Lee, Taewhoo, Yoon, Chanwoong, Hwang, Hyeon, Kang, Jaewoo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OLAPH: Improving Factuality in Biomedical Long-form Question Answering
by: Jeong, Minbyul, et al.
Published: (2024)

The Curious Case of Analogies: Investigating Analogical Reasoning in Large Language Models
by: Lee, Taewhoo, et al.
Published: (2025)

Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information
by: Park, Yein, et al.
Published: (2025)

ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains
by: Park, Yein, et al.
Published: (2024)

CompAct: Compressing Retrieved Documents Actively for Question Answering
by: Yoon, Chanwoong, et al.
Published: (2024)

Assessing LLM Reasoning Steps via Principal Knowledge Grounding
by: Hwang, Hyeon, et al.
Published: (2025)

ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage
by: Lee, Taewhoo, et al.
Published: (2024)

Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks
by: Kim, Hyunjae, et al.
Published: (2024)

LAPIS: Language Model-Augmented Police Investigation System
by: Kim, Heedou, et al.
Published: (2024)

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
by: Su, Zunhai, et al.
Published: (2025)

Systematic Outliers in Large Language Models
by: An, Yongqi, et al.
Published: (2025)

LittleBit: Ultra Low-Bit Quantization via Latent Factorization
by: Lee, Banseok, et al.
Published: (2025)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
by: Zhou, Chenxi, et al.
Published: (2025)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
by: Hu, Xing, et al.
Published: (2024)

SplitQuant: Layer Splitting for Low-Bit Neural Network Quantization
by: Song, Jaewoo, et al.
Published: (2025)

AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
by: Lee, Sangjun, et al.
Published: (2025)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
by: Chen, Mengzhao, et al.
Published: (2024)

SiLQ: Simple Large Language Model Quantization-Aware Training
by: Esser, Steven K., et al.
Published: (2025)

Interactions Across Blocks in Post-Training Quantization of Large Language Models
by: Shabanovi, Khasmamad, et al.
Published: (2024)

Language over Content: Tracing Cultural Understanding in Multilingual Large Language Models
by: Cho, Seungho, et al.
Published: (2025)

OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
by: Gadhikar, Advait, et al.
Published: (2025)

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs
by: Song, Jaewoo, et al.
Published: (2025)

Efficient Process Reward Modeling via Contrastive Mutual Information
by: Lee, Nakyung, et al.
Published: (2026)

Learning Dynamics in Continual Pre-Training for Large Language Models
by: Wang, Xingjin, et al.
Published: (2025)

Subgraph-level Universal Prompt Tuning
by: Lee, Junhyun, et al.
Published: (2024)

Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task
by: Lee, Hoonick, et al.
Published: (2024)

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
by: Xiao, Guangxuan, et al.
Published: (2022)

ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack
by: Park, Yein, et al.
Published: (2025)

APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
by: Guan, Ziyi, et al.
Published: (2024)

ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models
by: Yoon, Junho, et al.
Published: (2025)

On the Compressibility of Quantized Large Language Models
by: Mao, Yu, et al.
Published: (2024)

DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues
by: Jang, Kyochul, et al.
Published: (2025)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
by: Abbes, Istabrak, et al.
Published: (2025)

Integrating Pre-trained Language Model into Neural Machine Translation
by: Hwang, Soon-Jae, et al.
Published: (2023)

What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
by: Lv, Keyu, et al.
Published: (2026)

R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
by: Chen, Jiayi, et al.
Published: (2025)

CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models
by: Kocak, Aysenur, et al.
Published: (2025)

Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
by: Paglieri, Davide, et al.
Published: (2024)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
by: Mao, Shizhuo, et al.
Published: (2025)

MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
by: Wang, Dongwei, et al.
Published: (2026)