Imagen de Portada

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Cheng, Daixuan, Gu, Yuxian, Huang, Shaohan, Bi, Junyu, Huang, Minlie, Wei, Furu
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Adapting Large Language Models to Domains via Reading Comprehension
por: Cheng, Daixuan, et al.
Publicado: (2023)

MiniLLM: On-Policy Distillation of Large Language Models
por: Gu, Yuxian, et al.
Publicado: (2023)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Towards Optimal Learning of Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Data Selection via Optimal Control for Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Computer Environments Elicit General Agentic Intelligence in LLMs
por: Cheng, Daixuan, et al.
Publicado: (2026)

Reasoning with Exploration: An Entropy Perspective
por: Cheng, Daixuan, et al.
Publicado: (2025)

Textual Aesthetics in Large Language Models
por: Jiang, Lingjie, et al.
Publicado: (2024)

Thinking Augmented Pre-training
por: Wang, Liang, et al.
Publicado: (2025)

On-Policy Context Distillation for Language Models
por: Ye, Tianzhu, et al.
Publicado: (2026)

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
por: Li, Haoran, et al.
Publicado: (2024)

On Domain-Adaptive Post-Training for Multimodal Large Language Models
por: Cheng, Daixuan, et al.
Publicado: (2024)

Direct Preference Knowledge Distillation for Large Language Models
por: Li, Yixing, et al.
Publicado: (2024)

Mixture of LoRA Experts
por: Wu, Xun, et al.
Publicado: (2024)

Online Experiential Learning for Language Models
por: Ye, Tianzhu, et al.
Publicado: (2026)

MH-MoE: Multi-Head Mixture-of-Experts
por: Huang, Shaohan, et al.
Publicado: (2024)

Generative Pseudo-Labeling for Pre-Ranking with LLMs
por: Bi, Junyu, et al.
Publicado: (2026)

Black-Box On-Policy Distillation of Large Language Models
por: Ye, Tianzhu, et al.
Publicado: (2025)

Kosmos-G: Generating Images in Context with Multimodal Large Language Models
por: Pan, Xichen, et al.
Publicado: (2023)

Multi-Head Mixture-of-Experts
por: Wu, Xun, et al.
Publicado: (2024)

The Era of Agentic Organization: Learning to Organize with Language Models
por: Chi, Zewen, et al.
Publicado: (2025)

HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition
por: Liu, Yuxuan, et al.
Publicado: (2024)

Reward Reasoning Model
por: Guo, Jiaxin, et al.
Publicado: (2025)

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
por: Li, Zongqian, et al.
Publicado: (2026)

Universal YOCO for Efficient Depth Scaling
por: Sun, Yutao, et al.
Publicado: (2026)

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation
por: Wu, Xun, et al.
Publicado: (2024)

You Only Cache Once: Decoder-Decoder Architectures for Language Models
por: Sun, Yutao, et al.
Publicado: (2024)

VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
por: Jiang, Lingjie, et al.
Publicado: (2025)

Multimodal Latent Language Modeling with Next-Token Diffusion
por: Sun, Yutao, et al.
Publicado: (2024)

Text Embeddings by Weakly-Supervised Contrastive Pre-training
por: Wang, Liang, et al.
Publicado: (2022)

HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
por: Yuan, Kun, et al.
Publicado: (2026)

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
por: Cheng, Jiale, et al.
Publicado: (2023)

Reinforcement Pre-Training
por: Dong, Qingxiu, et al.
Publicado: (2025)

On-Policy RL with Optimal Reward Baseline
por: Hao, Yaru, et al.
Publicado: (2025)

Language Model Decoding as Direct Metrics Optimization
por: Ji, Haozhe, et al.
Publicado: (2023)

Code Aesthetics with Agentic Reward Feedback
por: Xiao, Bang, et al.
Publicado: (2025)

Training Language Model to Critique for Better Refinement
por: Yu, Tianshu, et al.
Publicado: (2025)

SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models
por: Cheng, Jiale, et al.
Publicado: (2024)

ConcEPT: Concept-Enhanced Pre-Training for Language Models
por: Wang, Xintao, et al.
Publicado: (2024)

Context-DPO: Aligning Language Models for Context-Faithfulness
por: Bi, Baolong, et al.
Publicado: (2024)