Image de couverture de livre

DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yin, Ruokai, Li, Yuhang, Lee, Donghyun, Panda, Priyadarshini
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation
par: Li, Yuhang, et autres
Publié: (2026)

Memba: Membrane-driven Parameter-Efficient Fine-Tuning for Mamba
par: Lee, Donghyun, et autres
Publié: (2025)

GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration
par: Li, Yuhang, et autres
Publié: (2025)

TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction
par: Li, Yuhang, et autres
Publié: (2024)

TT-SNN: Tensor Train Decomposition for Efficient Spiking Neural Network Training
par: Lee, Donghyun, et autres
Publié: (2024)

MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network
par: Lee, Donghyun, et autres
Publié: (2025)

PacQ: A SIMT Microarchitecture for Efficient Dataflow in Hyper-asymmetric GEMMs
par: Yin, Ruokai, et autres
Publié: (2025)

ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition
par: Xiao, Shiting, et autres
Publié: (2024)

Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization
par: Li, Guanchen, et autres
Publié: (2025)

Workload-Balanced Pruning for Sparse Spiking Neural Networks
par: Yin, Ruokai, et autres
Publié: (2023)

LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networks
par: Yin, Ruokai, et autres
Publié: (2024)

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
par: Yin, Lu, et autres
Publié: (2023)

QuRL: Efficient Reinforcement Learning with Quantized Rollout
par: Li, Yuhang, et autres
Publié: (2026)

Do We Really Need a Large Number of Visual Prompts?
par: Kim, Youngeun, et autres
Publié: (2023)

LNPT: Label-free Network Pruning and Training
par: Xiao, Jinying, et autres
Publié: (2024)

CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning
par: Aggarwal, Shivam, et autres
Publié: (2023)

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
par: Lee, Donghyun, et autres
Publié: (2024)

Selection Plateau and a Sparsity-Dependent Hierarchy of Pruning Features
par: Li, Guangqi, et autres
Publié: (2026)

When In-memory Computing Meets Spiking Neural Networks -- A Perspective on Device-Circuit-System-and-Algorithm Co-design
par: Moitra, Abhishek, et autres
Publié: (2024)

Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs
par: Gao, Chang, et autres
Publié: (2025)

Joint Training Across Multiple Activation Sparsity Regimes
par: Wang, Haotian
Publié: (2026)

Post-Training Statistical Calibration for Higher Activation Sparsity
par: Chua, Vui Seng, et autres
Publié: (2024)

ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation
par: Bhattacharjee, Abhiroop, et autres
Publié: (2024)

FF-INT8: Efficient Forward-Forward DNN Training on Edge Devices with INT8 Precision
par: Ma, Jingxiao, et autres
Publié: (2025)

Resting Neurons, Active Insights: Robustifying Activation Sparsity in LLMs via Spontaneity
par: Xu, Haotian, et autres
Publié: (2025)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
par: Li, Jiaxi, et autres
Publié: (2026)

Weight Concentration Regularization for Improving Pruning Robustness Under High Sparsity
par: Yun, Vincent-Daniel, et autres
Publié: (2025)

Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
par: Yun, Vincent-Daniel, et autres
Publié: (2026)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
par: Fu, Yao, et autres
Publié: (2025)

SiameseDuo++: Active Learning from Data Streams with Dual Augmented Siamese Networks
par: Malialis, Kleanthis, et autres
Publié: (2025)

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
par: Haziza, Daniel, et autres
Publié: (2025)

DuoGNN: Topology-aware Graph Neural Network with Homophily and Heterophily Interaction-Decoupling
par: Mancini, K., et autres
Publié: (2024)

Dual-Stage Invariant Continual Learning under Extreme Visual Sparsity
par: Zhang, Rangya, et autres
Publié: (2026)

SpikePool: Event-driven Spiking Transformer with Pooling Attention
par: Lee, Donghyun, et autres
Publié: (2025)

Asymmetric Duos: Sidekicks Improve Uncertainty
par: Zhou, Tim G., et autres
Publié: (2025)

GenQ: Quantization in Low Data Regimes with Generative Synthetic Data
par: Li, Yuhang, et autres
Publié: (2023)

Spiking Transformer with Spatial-Temporal Attention
par: Lee, Donghyun, et autres
Publié: (2024)

MEADOW: Memory-efficient Dataflow and Data Packing for Low Power Edge LLMs
par: Moitra, Abhishek, et autres
Publié: (2025)

To 2:4 Sparsity and Beyond: Neuron-level Activation Function to Accelerate LLM Pre-Training
par: Madhyastha, Meghana, et autres
Publié: (2026)

Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff
par: Liu, Junchen, et autres
Publié: (2025)