Cover Image

An Efficient Training Algorithm for Models with Block-wise Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Ding, Zuo, Zhiqun, Khalili, Mohammad Mahdi
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Post-processing for Fair Regression via Explainable SVD
by: Zuo, Zhiqun, et al.
Published: (2025)

Individual Fairness In Strategic Classification
by: Zuo, Zhiqun, et al.
Published: (2026)

Lookahead Counterfactual Fairness
by: Zuo, Zhiqun, et al.
Published: (2024)

Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identification
by: Chhabra, Vishnu Kabir, et al.
Published: (2025)

AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features
by: Zhu, Xudong, et al.
Published: (2025)

From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
by: Zhu, Xudong, et al.
Published: (2025)

DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
by: Shing, Makoto, et al.
Published: (2025)

Learning under Imitative Strategic Behavior with Unforeseeable Outcomes
by: Xie, Tian, et al.
Published: (2024)

On the transferability of Sparse Autoencoders for interpreting compressed models
by: Gupte, Suchit, et al.
Published: (2025)

EcoSpa: Efficient Transformer Training with Coupled Sparsity
by: Xiao, Jinqi, et al.
Published: (2025)

Training Long-Context LLMs Efficiently via Chunk-wise Optimization
by: Li, Wenhao, et al.
Published: (2025)

Efficient Knowledge Deletion from Trained Models through Layer-wise Partial Machine Unlearning
by: Gogineni, Vinay Chakravarthi, et al.
Published: (2024)

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
by: Xiao, Qiao, et al.
Published: (2026)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
by: Li, Jiaxi, et al.
Published: (2026)

Split Federated Learning Architectures for High-Accuracy and Low-Delay Model Training
by: Papageorgiou, Yiannis, et al.
Published: (2026)

Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
by: Ilin, Ivan, et al.
Published: (2025)

Realizing Unaligned Block-wise Pruning for DNN Acceleration on Mobile Devices
by: Lee, Hayun, et al.
Published: (2024)

FOAM: Blocked State Folding for Memory-Efficient LLM Training
by: Wen, Ziqing, et al.
Published: (2025)

Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement
by: Yang, Shu, et al.
Published: (2025)

On the Interplay Between Sparsity and Training in Deep Reinforcement Learning
by: Davelouis, Fatima, et al.
Published: (2025)

Joint Training Across Multiple Activation Sparsity Regimes
by: Wang, Haotian
Published: (2026)

Post-Training Statistical Calibration for Higher Activation Sparsity
by: Chua, Vui Seng, et al.
Published: (2024)

Improving MLLM Training Efficiency via Stage-Aware Sparsity
by: Shi, Kean, et al.
Published: (2025)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
by: Park, Jongseok, et al.
Published: (2026)

Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models
by: Xiao, He, et al.
Published: (2025)

GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning
by: Murata, Naoki, et al.
Published: (2026)

Cognitive Chunking for Soft Prompts: Accelerating Compressor Learning via Block-wise Causal Masking
by: Liu, Guojie, et al.
Published: (2026)

Differentially Private Block-wise Gradient Shuffle for Deep Learning
by: Zagardo, David
Published: (2024)

LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models
by: Su, Yupeng, et al.
Published: (2024)

MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Training
by: Luo, Yang, et al.
Published: (2025)

Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices
by: Ahmed, Mohamed Aboelenien, et al.
Published: (2025)

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
by: Zhang, Longteng, et al.
Published: (2026)

Sparsity-Aware Evolution for Model Merging
by: Zhang, Huan, et al.
Published: (2026)

RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning
by: Hu, Junhao, et al.
Published: (2025)

Toward Efficient Permutation for Hierarchical N:M Sparsity on GPUs
by: Yu, Seungmin, et al.
Published: (2024)

Information-Theoretic Greedy Layer-wise Training for Traffic Sign Recognition
by: Lyu, Shuyan, et al.
Published: (2025)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
by: An, Tai, et al.
Published: (2025)

Post-Training Sparse Attention with Double Sparsity
by: Yang, Shuo, et al.
Published: (2024)

Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
by: Zhou, Qihui, et al.
Published: (2025)

Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training
by: Liu, Zeyu, et al.
Published: (2025)