Copertina

DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xiang, Jingyang, Zhang, Sai Qian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
di: Ashkboos, Saleh, et al.
Pubblicazione: (2024)

Getting Free Bits Back from Rotational Symmetries in LLMs
di: He, Jiajun, et al.
Pubblicazione: (2024)

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
di: Huang, Xijie, et al.
Pubblicazione: (2024)

OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
di: Gadhikar, Advait, et al.
Pubblicazione: (2025)

Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference
di: Yi, Ke, et al.
Pubblicazione: (2024)

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
di: Su, Zunhai, et al.
Pubblicazione: (2025)

Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free
di: Choi, Euntae, et al.
Pubblicazione: (2025)

ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
di: Thrash, Chayne, et al.
Pubblicazione: (2026)

Online Ensemble Learning for Sector Rotation: A Gradient-Free Framework
di: Miao, Jiaju, et al.
Pubblicazione: (2023)

AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation
di: Kim, Seonggon, et al.
Pubblicazione: (2026)

Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization
di: Xiao, Jinying, et al.
Pubblicazione: (2025)

Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space
di: Zhang, Xiang, et al.
Pubblicazione: (2025)

DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
di: Gu, Tianteng, et al.
Pubblicazione: (2025)

Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization
di: Chen, Yung-Chin, et al.
Pubblicazione: (2026)

Angular Steering: Behavior Control via Rotation in Activation Space
di: Vu, Hieu M., et al.
Pubblicazione: (2025)

Spherical Steering: Geometry-Aware Activation Rotation for Language Models
di: You, Zejia, et al.
Pubblicazione: (2026)

Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs
di: Rachmil, Oren, et al.
Pubblicazione: (2025)

Don't Break the Boundary: Continual Unlearning for OOD Detection Based on Free Energy Repulsion
di: Peng, Ningkang, et al.
Pubblicazione: (2026)

AutoDFP: Automatic Data-Free Pruning via Channel Similarity Reconstruction
di: Li, Siqi, et al.
Pubblicazione: (2024)

Pushing the Limits of Block Rotations in Post-Training Quantization
di: Sanjeet, Sai, et al.
Pubblicazione: (2026)

Advancing Diffusion Models: Alias-Free Resampling and Enhanced Rotational Equivariance
di: Anjum, Md Fahim
Pubblicazione: (2024)

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
di: Yao, Yuxuan, et al.
Pubblicazione: (2026)

Lightweight Learning for Grant-Free Activity Detection in Cell-Free Massive MIMO Networks
di: Elkeshawy, Ali, et al.
Pubblicazione: (2025)

Approximating Uniform Random Rotations by Two-Block Structured Hadamard Rotations in High Dimensions
di: Zilca, Tomer, et al.
Pubblicazione: (2026)

CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
di: Yin, Xiangyang, et al.
Pubblicazione: (2026)

On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders
di: Simon, Elana, et al.
Pubblicazione: (2026)

The Uniformly Rotated Mondrian Kernel
di: Osborne, Calvin, et al.
Pubblicazione: (2025)

Rotational Sampling: A Plug-and-Play Encoder for Rotation-Invariant 3D Molecular GNNs
di: Jin, Dian
Pubblicazione: (2025)

Data-Free Privacy-Preserving for LLMs via Model Inversion and Selective Unlearning
di: Zhou, Xinjie, et al.
Pubblicazione: (2026)

Deep Learning-Based Approach for User Activity Detection with Grant-Free Random Access in Cell-Free Massive MIMO
di: Elkeshawy, Ali, et al.
Pubblicazione: (2024)

Simple KNN-Based Outlier Detection Achieves Robust Clustering
di: Jiang, Tianle, et al.
Pubblicazione: (2026)

AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch
di: Yang, Max, et al.
Pubblicazione: (2024)

Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs
di: Kuchibhotla, Hari Chandana, et al.
Pubblicazione: (2025)

Deep-Unfolded Massive Grant-Free Transmission in Cell-Free Wireless Communication Systems
di: Sun, Gangle, et al.
Pubblicazione: (2025)

RotRNN: Modelling Long Sequences with Rotations
di: Biegun, Kai, et al.
Pubblicazione: (2024)

Change-of-Basis Pruning via Rotational Invariance
di: Ning, Alex, et al.
Pubblicazione: (2025)

Detecting Anomalies Using Rotated Isolation Forest
di: Monemizadeh, Vahideh, et al.
Pubblicazione: (2025)

Refining Covariance Matrix Estimation in Stochastic Gradient Descent Through Bias Reduction
di: Wei, Ziyang, et al.
Pubblicazione: (2026)

Constructing 3D Rotational Invariance and Equivariance with Symmetric Tensor Networks
di: Zhang, Meng, et al.
Pubblicazione: (2025)

SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Scheduling
di: Wang, Jinghui, et al.
Pubblicazione: (2025)