Copertina

S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Hu, Yuezhou, Zhu, Jun, Chen, Jianfei
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Accelerating Transformer Pre-training with 2:4 Sparsity
di: Hu, Yuezhou, et al.
Pubblicazione: (2024)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
di: Huang, Weiyu, et al.
Pubblicazione: (2025)

SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning
di: Zhang, Jintao, et al.
Pubblicazione: (2026)

SparseDM: Toward Sparse Efficient Diffusion Models
di: Wang, Kafeng, et al.
Pubblicazione: (2024)

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training
di: Huang, Weiyu, et al.
Pubblicazione: (2024)

Identifying Sensitive Weights via Post-quantization Integral
di: Hu, Yuezhou, et al.
Pubblicazione: (2025)

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism
di: Li, Guanchen, et al.
Pubblicazione: (2024)

Efficient Backpropagation with Variance-Controlled Adaptive Sampling
di: Wang, Ziteng, et al.
Pubblicazione: (2024)

Communication Efficient LLM Pre-training with SparseLoCo
di: Sarfi, Amir, et al.
Pubblicazione: (2025)

Towards a General Framework for Continual Learning with Pre-training
di: Wang, Liyuan, et al.
Pubblicazione: (2023)

Deterministic Differentiable Structured Pruning for Large Language Models
di: Huang, Weiyu, et al.
Pubblicazione: (2026)

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
di: Thangarasa, Vithursan, et al.
Pubblicazione: (2024)

Efficient Continual Pre-training of LLMs for Low-resource Languages
di: Nag, Arijit, et al.
Pubblicazione: (2024)

Efficient Hyperparameter Tuning via Trajectory Invariance Principle
di: Li, Bingrui, et al.
Pubblicazione: (2025)

ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
di: Wang, Ziteng, et al.
Pubblicazione: (2024)

Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
di: Sui, Yueyuan, et al.
Pubblicazione: (2026)

Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks
di: Zhan, Qishi, et al.
Pubblicazione: (2026)

Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization
di: Xi, Haocheng, et al.
Pubblicazione: (2024)

LOST: Low-rank and Sparse Pre-training for Large Language Models
di: Li, Jiaxi, et al.
Pubblicazione: (2025)

Structural Pruning of Pre-trained Language Models via Neural Architecture Search
di: Klein, Aaron, et al.
Pubblicazione: (2024)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
di: Hu, Yuezhou, et al.
Pubblicazione: (2025)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
di: Xia, Mengzhou, et al.
Pubblicazione: (2023)

RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
di: Wei, Quan, et al.
Pubblicazione: (2025)

Practical Continual Forgetting for Pre-trained Vision Models
di: Zhao, Hongbo, et al.
Pubblicazione: (2025)

CLDyB: Towards Dynamic Benchmarking for Continual Learning with Pre-trained Models
di: Chen, Shengzhuang, et al.
Pubblicazione: (2025)

PAODING: A High-fidelity Data-free Pruning Toolkit for Debloating Pre-trained Neural Networks
di: Meng, Mark Huasong, et al.
Pubblicazione: (2024)

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
di: Liu, Enshu, et al.
Pubblicazione: (2024)

Pruning for Sparse Diffusion Models based on Gradient Flow
di: Wan, Ben, et al.
Pubblicazione: (2025)

PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning
di: Ying, Chengyang, et al.
Pubblicazione: (2024)

Caption, Create, Continue: Continual Learning with Pre-trained Generative Vision-Language Models
di: Solomon, Indu, et al.
Pubblicazione: (2024)

Oscillation-Reduced MXFP4 Training for Vision Transformers
di: Chen, Yuxiang, et al.
Pubblicazione: (2025)

Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs
di: Zheng, Kaiwen, et al.
Pubblicazione: (2023)

Sparse Variational Contaminated Noise Gaussian Process Regression with Applications in Geomagnetic Perturbations Forecasting
di: Iong, Daniel, et al.
Pubblicazione: (2024)

Towards Pre-trained Graph Condensation via Optimal Transport
di: Yan, Yeyu, et al.
Pubblicazione: (2025)

An Efficient Replay for Class-Incremental Learning with Pre-trained Models
di: Yin, Weimin, et al.
Pubblicazione: (2024)

CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search
di: Park, Cheonjun
Pubblicazione: (2026)

JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates
di: Yi, Kai, et al.
Pubblicazione: (2026)

Layerwise Progressive Freezing Enables STE-Free Training of Deep Binary Neural Networks
di: Smith, Evan Gibson, et al.
Pubblicazione: (2026)

S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs
di: Wang, Yuhan, et al.
Pubblicazione: (2026)

Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs
di: Gao, Chang, et al.
Pubblicazione: (2025)