Copertina

IMU-1: Sample-Efficient Pre-training of Small Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Grigorev, George
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
di: Yi, Rongjie, et al.
Pubblicazione: (2024)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
di: Samragh, Mohammad, et al.
Pubblicazione: (2024)

An Efficient Replay for Class-Incremental Learning with Pre-trained Models
di: Yin, Weimin, et al.
Pubblicazione: (2024)

Pre-trained Large Language Models Learn Hidden Markov Models In-context
di: Dai, Yijia, et al.
Pubblicazione: (2025)

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
di: Zhou, Jiecheng, et al.
Pubblicazione: (2025)

Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model
di: Niu, Peisong, et al.
Pubblicazione: (2025)

Investigating Data Contamination for Pre-training Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2024)

Aligning Pre-trained Models for Spoken Language Translation
di: Sedláček, Šimon, et al.
Pubblicazione: (2024)

Sequence-to-Sequence Spanish Pre-trained Language Models
di: Araujo, Vladimir, et al.
Pubblicazione: (2023)

Mochi: Aligning Pre-training and Inference for Efficient Graph Foundation Models via Meta-Learning
di: Mattos, João, et al.
Pubblicazione: (2026)

Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models
di: Wen, Ziting, et al.
Pubblicazione: (2024)

FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics
di: Duan, ChenRui, et al.
Pubblicazione: (2024)

Integrating Pre-trained Language Model into Neural Machine Translation
di: Hwang, Soon-Jae, et al.
Pubblicazione: (2023)

OSF: On Pre-training and Scaling of Sleep Foundation Models
di: Shuai, Zitao, et al.
Pubblicazione: (2026)

Transfer Learning with Pre-trained Conditional Generative Models
di: Yamaguchi, Shin'ya, et al.
Pubblicazione: (2022)

Scaling Laws for Pre-training Agents and World Models
di: Pearce, Tim, et al.
Pubblicazione: (2024)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
di: Schröder, Christopher, et al.
Pubblicazione: (2024)

Pre-training a Transformer-Based Generative Model Using a Small Sepedi Dataset
di: Ramalepe, Simon P., et al.
Pubblicazione: (2025)

A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learning
di: Gai, Keke, et al.
Pubblicazione: (2025)

Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model
di: Jin, Jiarui, et al.
Pubblicazione: (2025)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
di: Ibrahim, Adam, et al.
Pubblicazione: (2024)

Pre-training Limited Memory Language Models with Internal and External Knowledge
di: Zhao, Linxi, et al.
Pubblicazione: (2025)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
di: Song, Weixi, et al.
Pubblicazione: (2023)

MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
di: Lin, Xi Victoria, et al.
Pubblicazione: (2024)

Heterogeneous Graph Pre-training Based Model for Secure and Efficient Prediction of Default Risk Propagation among Bond Issuers
di: Li, Xurui, et al.
Pubblicazione: (2025)

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing
di: Toral, Lukas, et al.
Pubblicazione: (2025)

Graph Generative Pre-trained Transformer
di: Chen, Xiaohui, et al.
Pubblicazione: (2025)

On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models
di: Farhat, Sean, et al.
Pubblicazione: (2024)

Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Series
di: Zhang, Weijia, et al.
Pubblicazione: (2024)

Priming: Hybrid State Space Models From Pre-trained Transformers
di: Chattopadhyay, Aditya, et al.
Pubblicazione: (2026)

A Pre-trained Data Deduplication Model based on Active Learning
di: Shi, Haochen, et al.
Pubblicazione: (2023)

HiFloat4 Format for Language Model Pre-training on Ascend NPUs
di: Taghian, Mehran, et al.
Pubblicazione: (2026)

CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model
di: Chiang, Shang-Hsuan, et al.
Pubblicazione: (2024)

CASE: Efficient Curricular Data Pre-training for Building Assistive Psychology Expert Models
di: Harne, Sarthak, et al.
Pubblicazione: (2024)

Machine Unlearning of Pre-trained Large Language Models
di: Yao, Jin, et al.
Pubblicazione: (2024)

The Future of Large Language Model Pre-training is Federated
di: Sani, Lorenzo, et al.
Pubblicazione: (2024)

Revisiting Pre-trained Language Models for Vulnerability Detection
di: Li, Youpeng, et al.
Pubblicazione: (2025)

Efficient Online RL Fine Tuning with Offline Pre-trained Policy Only
di: Xiao, Wei, et al.
Pubblicazione: (2025)

Stabilizing RNN Gradients through Pre-training
di: Herranz-Celotti, Luca, et al.
Pubblicazione: (2023)

An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibration
di: Naganuma, Hiroki, et al.
Pubblicazione: (2023)