Image de couverture de livre

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Boqian, Xiao, Qiao, Okanovic, Patrik, Sternal, Tomasz, van Keulen, Maurice, Pechenizkiy, Mykola, Mocanu, Elena, Hoefler, Torsten, Mocanu, Decebal Constantin
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
par: Xiao, Qiao, et autres
Publié: (2026)

Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness
par: Wu, Boqian, et autres
Publié: (2024)

E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation
par: Wu, Boqian, et autres
Publié: (2023)

Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity
par: Xiao, Qiao, et autres
Publié: (2025)

Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution
par: Xiao, Qiao, et autres
Publié: (2025)

Are Sparse Neural Networks Better Hard Sample Learners?
par: Xiao, Qiao, et autres
Publié: (2024)

Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
par: Atashgahi, Zahra, et autres
Publié: (2023)

NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling
par: Grooten, Bram, et autres
Publié: (2025)

Unveiling the Power of Sparse Neural Networks for Feature Selection
par: Atashgahi, Zahra, et autres
Publié: (2024)

Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
par: Liu, Shiwei, et autres
Publié: (2021)

Sparse-to-Sparse Training of Diffusion Models
par: Oliveira, Inês Cardoso, et autres
Publié: (2025)

Dynamic Data Pruning for Automatic Speech Recognition
par: Xiao, Qiao, et autres
Publié: (2024)

Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsity
par: Muslimani, Calarina, et autres
Publié: (2024)

EntryPrune: Neural Network Feature Selection using First Impressions
par: Zimmer, Felix, et autres
Publié: (2024)

Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
par: Wesselink, Wieger, et autres
Publié: (2025)

Confounder Detection via Treatment Intent: A New Observational Study Design
par: Plecko, Drago, et autres
Publié: (2026)

You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
par: Huang, Tianjin, et autres
Publié: (2022)

Self-Regulated Neurogenesis for Online Data-Incremental Learning
par: Yildirim, Murat Onur, et autres
Publié: (2024)

Epidemiology of Large Language Models: A Benchmark for Observational Distribution Knowledge
par: Plecko, Drago, et autres
Publié: (2025)

Process Reward Agents for Steering Knowledge-Intensive Reasoning
par: Sohn, Jiwoong, et autres
Publié: (2026)

Large Language Model Selection with Limited Annotations
par: Durmazkeser, Yavuz, et autres
Publié: (2026)

Active Model Selection for Large Language Models
par: Durmazkeser, Yavuz, et autres
Publié: (2025)

LiMTR: Time Series Motion Prediction for Diverse Road Users through Multimodal Feature Integration
par: Oerlemans, Camiel, et autres
Publié: (2024)

Dificultăți emoționale și psihosociale ale veteranilor din teatrele de operații
par: Mocanu, Sorin, et autres
Publié: (2026)

Generalizations of four hyperbolic-type metrics and Gromov hyperbolicity
par: Mocanu, Marcelina
Publié: (2024)

High Performance Unstructured SpMM Computation Using Tensor Cores
par: Okanovic, Patrik, et autres
Publié: (2024)

All models are wrong, some are useful: Model Selection with Limited Labels
par: Okanovic, Patrik, et autres
Publié: (2024)

Nerva: a Truly Sparse Implementation of Neural Networks
par: Wesselink, Wieger, et autres
Publié: (2024)

Fertility preservation in cancer patients: Data collection, analysis, and continuous improvement of cryostorage
par: Edgar Mocanu, et autres
Publié: (2025)

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
par: Li, Shigang, et autres
Publié: (2021)

BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers
par: Okanovic, Patrik, et autres
Publié: (2025)

Local points on twists of $X(p)$ with applications
par: Freitas, Nuno, et autres
Publié: (2025)

English in Spain: Education, attitudes and native‐speakerism
par: Enric Llurda, et autres
Publié: (2024)

SpComm3D: A Framework for Enabling Sparse Communication in 3D Sparse Kernels
par: Abubaker, Nabil, et autres
Publié: (2024)

Near-Optimal Sparse Allreduce for Distributed Deep Learning
par: Li, Shigang, et autres
Publié: (2022)

The Stochastic-Dissipative Störmer Problem-Trajectories and Radiation Patterns
par: Harko, Tiberiu, et autres
Publié: (2025)

The Stochastic‐Dissipative Störmer Problem‐Trajectories and Radiation Patterns
par: Tiberiu Harko, et autres
Publié: (2025)

Everyone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data
par: Raes, Rik, et autres
Publié: (2024)

Self-Supervised Learning to Fly using Efficient Semantic Segmentation and Metric Depth Estimation for Low-Cost Autonomous UAVs
par: Mocanu, Sebastian, et autres
Publié: (2025)

The Ptolemy-Alhazen problem and quadric surface mirror reflection
par: Fujimura, Masayo, et autres
Publié: (2021)