Image de couverture de livre

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liu, Hongyi, Saha, Rajarshi, Jia, Zhen, Park, Youngsuk, Huang, Jiaji, Sabach, Shoham, Wang, Yu-Xiang, Karypis, George
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs
par: Liu, Hongyi, et autres
Publié: (2025)

A Proximal Operator for Inducing 2:4-Sparsity
par: Kübler, Jonas M, et autres
Publié: (2025)

Dynamic FISTA for Convex Composite Bi-Level Optimization
par: Merchav, Roey, et autres
Publié: (2024)

Inference Optimization of Foundation Models on AI Accelerators
par: Park, Youngsuk, et autres
Publié: (2024)

Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
par: Mavromatis, Costas, et autres
Publié: (2024)

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
par: Liu, Zuxin, et autres
Publié: (2023)

Training LLMs with MXFP4
par: Tseng, Albert, et autres
Publié: (2025)

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
par: Jiang, Ruichen, et autres
Publié: (2024)

Directional-Clamp PPO
par: Karpel, Gilad, et autres
Publié: (2025)

Displacement-Resistant Extensions of DPO with Nonconvex $f$-Divergences
par: Pipano, Idan, et autres
Publié: (2026)

Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
par: Zhang, Di, et autres
Publié: (2026)

TritonRL: Training LLMs to Think and Code Triton Without Cheating
par: Woo, Jiin, et autres
Publié: (2025)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
par: Fang, Gongfan, et autres
Publié: (2024)

Silent Until Sparse: Backdoor Attacks on Semi-Structured Sparsity
par: Guo, Wei, et autres
Publié: (2025)

Learning the Target Network in Function Space
par: Asadi, Kavosh, et autres
Publié: (2024)

Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
par: Bian, Song, et autres
Publié: (2025)

Inducing Semi-Structured Sparsity by Masking for Efficient Model Inference in Convolutional Networks
par: Danhofer, David A.
Publié: (2024)

Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
par: Deng, Wenlong, et autres
Publié: (2026)

SemPool: Simple, robust, and interpretable KG pooling for enhancing language models
par: Mavromatis, Costas, et autres
Publié: (2024)

Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learning
par: Meinhardt, Georg, et autres
Publié: (2024)

Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularization
par: Pang, Jing-Cheng, et autres
Publié: (2021)

Comparing Few to Rank Many: Active Human Preference Learning using Randomized Frank-Wolfe
par: Thekumparampil, Kiran Koshy, et autres
Publié: (2024)

Extending Input Contexts of Language Models through Training on Segmented Sequences
par: Karypis, Petros, et autres
Publié: (2023)

FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
par: Deb, Rohan, et autres
Publié: (2025)

Sparsity via Sparse Group $k$-max Regularization
par: Tao, Qinghua, et autres
Publié: (2024)

Stochastic Rounding for LLM Training: Theory and Practice
par: Ozkara, Kaan, et autres
Publié: (2025)

On the Convergence of FedProx with Extrapolation and Inexact Prox
par: Li, Hanmin, et autres
Publié: (2024)

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning
par: Mavromatis, Costas, et autres
Publié: (2024)

Dynamic Sparse Training with Structured Sparsity
par: Lasby, Mike, et autres
Publié: (2023)

Differentially Private Bias-Term Fine-tuning of Foundation Models
par: Bu, Zhiqi, et autres
Publié: (2022)

Analysis of Kernel Mirror Prox for Measure Optimization
par: Dvurechensky, Pavel, et autres
Publié: (2024)

Prox-NAG-GS: A Semi-Implicit Proximal Method for Composite Optimization
par: Wiykiynyuy, Sikeh Gisele, et autres
Publié: (2026)

Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains
par: Viano, Luca, et autres
Publié: (2026)

MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
par: Ozkara, Kaan, et autres
Publié: (2024)

MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on LLMs
par: Sun, Yan, et autres
Publié: (2025)

Tissue-Contrastive Semi-Masked Autoencoders for Segmentation Pretraining on Chest CT
par: Zheng, Jie, et autres
Publié: (2024)

Improving Robustness In Sparse Autoencoders via Masked Regularization
par: Narayanaswamy, Vivek, et autres
Publié: (2026)

Crisp Attention: Regularizing Transformers via Structured Sparsity
par: Gandhi, Sagar, et autres
Publié: (2025)

Locality Regularized Reconstruction: Structured Sparsity and Delaunay Triangulations
par: Mueller, Marshall, et autres
Publié: (2024)

Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution
par: Xiao, Qiao, et autres
Publié: (2025)