Image de couverture de livre

Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Diao, Shizhe, Yang, Yu, Fu, Yonggan, Dong, Xin, Su, Dan, Kliegl, Markus, Chen, Zijia, Belcak, Peter, Suhara, Yoshi, Yin, Hongxu, Patwary, Mostofa, Yingyan, Lin, Kautz, Jan, Molchanov, Pavlo
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Small Language Models are the Future of Agentic AI
par: Belcak, Peter, et autres
Publié: (2025)

Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
par: Taghibakhshi, Ali, et autres
Publié: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
par: Dong, Xin, et autres
Publié: (2024)

Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset
par: Su, Dan, et autres
Publié: (2024)

Universal Deep Research: Bring Your Own Model and Strategy
par: Belcak, Peter, et autres
Publié: (2025)

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
par: Fu, Yonggan, et autres
Publié: (2025)

Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
par: Belcak, Peter, et autres
Publié: (2025)

LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement
par: Ye, Zhifan, et autres
Publié: (2025)

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
par: Su, Hongjin, et autres
Publié: (2025)

VILA: On Pre-training for Visual Language Models
par: Lin, Ji, et autres
Publié: (2023)

LLM Pruning and Distillation in Practice: The Minitron Approach
par: Sreenivas, Sharath Turuvekere, et autres
Publié: (2024)

Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
par: Taghibakhshi, Ali, et autres
Publié: (2025)

Compact Language Models via Pruning and Knowledge Distillation
par: Muralidharan, Saurav, et autres
Publié: (2024)

Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
par: Mahabadi, Rabeeh Karimi, et autres
Publié: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
par: Fu, Yonggan, et autres
Publié: (2025)

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
par: Liu, Shih-Yang, et autres
Publié: (2026)

Source Identification in Abstractive Summarization
par: Suhara, Yoshi, et autres
Publié: (2024)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
par: Shi, Dachuan, et autres
Publié: (2025)

Fast-dLLM v2: Efficient Block-Diffusion LLM
par: Wu, Chengyue, et autres
Publié: (2025)

Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
par: Akter, Syeda Nahida, et autres
Publié: (2025)

LITA: Language Instructed Temporal-Localization Assistant
par: Huang, De-An, et autres
Publié: (2024)

Flextron: Many-in-One Flexible Large Language Model
par: Cai, Ruisi, et autres
Publié: (2024)

AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One
par: Ranzinger, Mike, et autres
Publié: (2023)

Noisy Pairing and Partial Supervision for Stylized Opinion Summarization
par: Iso, Hayate, et autres
Publié: (2022)

Large Language Models are Inconsistent and Biased Evaluators
par: Stureborg, Rickard, et autres
Publié: (2024)

ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
par: Wang, Zhilin, et autres
Publié: (2025)

FasterViT: Fast Vision Transformers with Hierarchical Attention
par: Hatamizadeh, Ali, et autres
Publié: (2023)

$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
par: Du, Zhenbang, et autres
Publié: (2026)

Scaling Vision Pre-Training to 4K Resolution
par: Shi, Baifeng, et autres
Publié: (2025)

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks
par: Bair, Anna, et autres
Publié: (2023)

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
par: Heinrich, Greg, et autres
Publié: (2024)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
par: Fang, Gongfan, et autres
Publié: (2024)

When2Call: When (not) to Call Tools
par: Ross, Hayley, et autres
Publié: (2025)

BroRL: Scaling Reinforcement Learning via Broadened Exploration
par: Hu, Jian, et autres
Publié: (2025)

TiDAR: Think in Diffusion, Talk in Autoregression
par: Liu, Jingyu, et autres
Publié: (2025)

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
par: Liu, Shih-Yang, et autres
Publié: (2025)

Nemotron-4 15B Technical Report
par: Parmar, Jupinder, et autres
Publié: (2024)

Step Out and Seek Around: On Warm-Start Training with Incremental Data
par: Shen, Maying, et autres
Publié: (2024)

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning
par: Kang, Minki, et autres
Publié: (2026)

Entropy-Regularized Process Reward Model
par: Zhang, Hanning, et autres
Publié: (2024)