Image de couverture de livre

Joint Training Across Multiple Activation Sparsity Regimes

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Wang, Haotian
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy
par: Balagansky, Nikita, et autres
Publié: (2025)

Post-Training Statistical Calibration for Higher Activation Sparsity
par: Chua, Vui Seng, et autres
Publié: (2024)

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
par: Haziza, Daniel, et autres
Publié: (2025)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
par: Li, Jiaxi, et autres
Publié: (2026)

Towards the Connection between Activation Sparsity and Flat Minima
par: Peng, Ze, et autres
Publié: (2026)

Activation Sparsity Opportunities for Compressing General Large Language Models
par: Dhar, Nobel, et autres
Publié: (2024)

Motivating Next-Gen Accelerators with Flexible (N:M) Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches
par: Alanova, Shirin, et autres
Publié: (2025)

WiSparse: Boosting LLM Inference Efficiency with Weight-Aware Mixed Activation Sparsity
par: Chen, Lei, et autres
Publié: (2026)

Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers
par: Wild, Cody, et autres
Publié: (2024)

On the Interplay Between Sparsity and Training in Deep Reinforcement Learning
par: Davelouis, Fatima, et autres
Publié: (2025)

An Efficient Training Algorithm for Models with Block-wise Sparsity
par: Zhu, Ding, et autres
Publié: (2025)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
par: Park, Jongseok, et autres
Publié: (2026)

Improving MLLM Training Efficiency via Stage-Aware Sparsity
par: Shi, Kean, et autres
Publié: (2025)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
par: An, Tai, et autres
Publié: (2025)

Improving Decision Sparsity
par: Sun, Yiyang, et autres
Publié: (2024)

EcoSpa: Efficient Transformer Training with Coupled Sparsity
par: Xiao, Jinqi, et autres
Publié: (2025)

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
par: Xiao, Qiao, et autres
Publié: (2026)

Post-Training Sparse Attention with Double Sparsity
par: Yang, Shuo, et autres
Publié: (2024)

Homeostasis and Sparsity in Transformer
par: Kotyuzanskiy, Leonid, et autres
Publié: (2024)

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
par: Shrestha, Susav, et autres
Publié: (2025)

Scaling Attention via Feature Sparsity
par: Xie, Yan, et autres
Publié: (2026)

EvoXplain: When Machine Learning Models Agree on Predictions but Disagree on Why -- Measuring Mechanistic Multiplicity Across Training Runs
par: Bensmail, Chama
Publié: (2025)

Sparsity and Out-of-Distribution Generalization
par: Aaronson, Scott, et autres
Publié: (2026)

Sparsity and Superposition in Mixture of Experts
par: Chaudhari, Marmik, et autres
Publié: (2025)

Compositional Sparsity as an Inductive Bias for Neural Architecture Design
par: Lin, Hongyu, et autres
Publié: (2026)

HPO: Hysteretic Policy Optimization for Stable and Efficient Training under Sparse-Reward Regime
par: Sana, Mohamed, et autres
Publié: (2026)

Sparsity-Aware Evolution for Model Merging
par: Zhang, Huan, et autres
Publié: (2026)

On the Sparsity of the Strong Lottery Ticket Hypothesis
par: Natale, Emanuele, et autres
Publié: (2024)

On the Identifiability of Nonlinear ICA: Sparsity and Beyond
par: Zheng, Yujia, et autres
Publié: (2022)

Wasserstein Distances, Neuronal Entanglement, and Sparsity
par: Sawmya, Shashata, et autres
Publié: (2024)

Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches
par: Chen, Yifang, et autres
Publié: (2025)

Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments
par: Schweisthal, Jonas, et autres
Publié: (2024)

Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
par: Cui, Brandon, et autres
Publié: (2026)

Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts
par: Chu, Mengdi, et autres
Publié: (2026)

Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning
par: Ma, Guozheng, et autres
Publié: (2025)

LoRA Dropout as a Sparsity Regularizer for Overfitting Control
par: Lin, Yang, et autres
Publié: (2024)

Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
par: Zhang, Te, et autres
Publié: (2025)

Inference Time Context Sparsity: Illusion or Opportunity?
par: Joshi, Sahil, et autres
Publié: (2026)

HashAttention: Semantic Sparsity for Faster Inference
par: Desai, Aditya, et autres
Publié: (2024)

Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization
par: Chen, Xi, et autres
Publié: (2026)