Copertina

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xiao, Qiao, Wu, Boqian, Okanovic, Patrik, Sternal, Tomasz, van Keulen, Maurice, Mocanu, Elena, Pechenizkiy, Mykola, Mocanu, Decebal Constantin, Hoefler, Torsten
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
di: Wu, Boqian, et al.
Pubblicazione: (2026)

Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness
di: Wu, Boqian, et al.
Pubblicazione: (2024)

E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation
di: Wu, Boqian, et al.
Pubblicazione: (2023)

Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
di: Atashgahi, Zahra, et al.
Pubblicazione: (2023)

Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity
di: Xiao, Qiao, et al.
Pubblicazione: (2025)

Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution
di: Xiao, Qiao, et al.
Pubblicazione: (2025)

Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
di: Liu, Shiwei, et al.
Pubblicazione: (2021)

Are Sparse Neural Networks Better Hard Sample Learners?
di: Xiao, Qiao, et al.
Pubblicazione: (2024)

NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling
di: Grooten, Bram, et al.
Pubblicazione: (2025)

Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsity
di: Muslimani, Calarina, et al.
Pubblicazione: (2024)

Unveiling the Power of Sparse Neural Networks for Feature Selection
di: Atashgahi, Zahra, et al.
Pubblicazione: (2024)

Dynamic Data Pruning for Automatic Speech Recognition
di: Xiao, Qiao, et al.
Pubblicazione: (2024)

Sparse-to-Sparse Training of Diffusion Models
di: Oliveira, Inês Cardoso, et al.
Pubblicazione: (2025)

EntryPrune: Neural Network Feature Selection using First Impressions
di: Zimmer, Felix, et al.
Pubblicazione: (2024)

Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
di: Wesselink, Wieger, et al.
Pubblicazione: (2025)

Confounder Detection via Treatment Intent: A New Observational Study Design
di: Plecko, Drago, et al.
Pubblicazione: (2026)

You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
di: Huang, Tianjin, et al.
Pubblicazione: (2022)

Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledge
di: Plecko, Drago, et al.
Pubblicazione: (2025)

Self-Regulated Neurogenesis for Online Data-Incremental Learning
di: Yildirim, Murat Onur, et al.
Pubblicazione: (2024)

Process Reward Agents for Steering Knowledge-Intensive Reasoning
di: Sohn, Jiwoong, et al.
Pubblicazione: (2026)

Large Language Model Selection with Limited Annotations
di: Durmazkeser, Yavuz, et al.
Pubblicazione: (2026)

Active Model Selection for Large Language Models
di: Durmazkeser, Yavuz, et al.
Pubblicazione: (2025)

LiMTR: Time Series Motion Prediction for Diverse Road Users through Multimodal Feature Integration
di: Oerlemans, Camiel, et al.
Pubblicazione: (2024)

Dificultăți emoționale și psihosociale ale veteranilor din teatrele de operații
di: Mocanu, Sorin, et al.
Pubblicazione: (2026)

Generalizations of four hyperbolic-type metrics and Gromov hyperbolicity
di: Mocanu, Marcelina
Pubblicazione: (2024)

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
di: Li, Shigang, et al.
Pubblicazione: (2021)

High Performance Unstructured SpMM Computation Using Tensor Cores
di: Okanovic, Patrik, et al.
Pubblicazione: (2024)

All models are wrong, some are useful: Model Selection with Limited Labels
di: Okanovic, Patrik, et al.
Pubblicazione: (2024)

Local points on twists of $X(p)$ with applications
di: Freitas, Nuno, et al.
Pubblicazione: (2025)

English in Spain: Education, attitudes and native‐speakerism
di: Enric Llurda, et al.
Pubblicazione: (2024)

Self-Supervised Learning to Fly using Efficient Semantic Segmentation and Metric Depth Estimation for Low-Cost Autonomous UAVs
di: Mocanu, Sebastian, et al.
Pubblicazione: (2025)

Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Sampling
di: Provodin, Danil, et al.
Pubblicazione: (2024)

The Stochastic-Dissipative Störmer Problem-Trajectories and Radiation Patterns
di: Harko, Tiberiu, et al.
Pubblicazione: (2025)

Fertility preservation in cancer patients: Data collection, analysis, and continuous improvement of cryostorage
di: Edgar Mocanu, et al.
Pubblicazione: (2025)

The Stochastic‐Dissipative Störmer Problem‐Trajectories and Radiation Patterns
di: Tiberiu Harko, et al.
Pubblicazione: (2025)

The Ptolemy-Alhazen problem and quadric surface mirror reflection
di: Fujimura, Masayo, et al.
Pubblicazione: (2021)

Non-trivial Integer Solutions of $x^r+y^r=Dz^p$
di: Kara, Yasemin, et al.
Pubblicazione: (2024)

Barrlund's distance function and quasiconformal maps
di: Fujimura, Masayo, et al.
Pubblicazione: (2019)

HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents
di: Tomilin, Tristan, et al.
Pubblicazione: (2025)

Everyone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data
di: Raes, Rik, et al.
Pubblicazione: (2024)