Imagen de Portada

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Shaobo, Ouyang, Xuan, Xu, Tianyi, Hu, Yuzheng, Liu, Jialin, Chen, Guo, Zhang, Tianyu, Zheng, Junhao, Yang, Kexin, Ren, Xingzhang, Liu, Dayiheng, Zhang, Linfeng
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
por: Wang, Shaobo, et al.
Publicado: (2026)

An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
por: Tian, Yuxin, et al.
Publicado: (2024)

Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
por: Zhou, Yufa, et al.
Publicado: (2025)

Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
por: Wang, Shaobo, et al.
Publicado: (2025)

DataMan: Data Manager for Pre-training Large Language Models
por: Peng, Ru, et al.
Publicado: (2025)

RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
por: Bi, Baolong, et al.
Publicado: (2025)

On Predicting the Post-training Potential of Pre-trained LLMs
por: Li, Xiaoyuan, et al.
Publicado: (2026)

Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
por: Wang, Shaobo, et al.
Publicado: (2025)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
por: Wang, Shaobo, et al.
Publicado: (2026)

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
por: Yang, An, et al.
Publicado: (2024)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
por: Tang, Shengkun, et al.
Publicado: (2026)

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
por: Zhou, Jiecheng, et al.
Publicado: (2025)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
por: Zhou, Fan, et al.
Publicado: (2024)

Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis
por: Jiao, Zhengbo, et al.
Publicado: (2026)

Towards Principled Dataset Distillation: A Spectral Distribution Perspective
por: Wu, Ruixi, et al.
Publicado: (2026)

EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision
por: Xu, Tianyi, et al.
Publicado: (2026)

Towards Efficient LLMs Annealing with Principled Sample Selection
por: Xu, Yuanjian, et al.
Publicado: (2026)

How to Set the Batch Size for Large-Scale Pre-training?
por: Zhou, Yunhua, et al.
Publicado: (2026)

OPUS: Occupancy Prediction Using a Sparse Set
por: Wang, Jiabao, et al.
Publicado: (2024)

GraphGPT: Generative Pre-trained Graph Eulerian Transformer
por: Zhao, Qifang, et al.
Publicado: (2023)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
por: Qian, Chen, et al.
Publicado: (2024)

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
por: Ren, Xuan, et al.
Publicado: (2025)

Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
por: He, Yu, et al.
Publicado: (2025)

Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling
por: Chen, Yujie, et al.
Publicado: (2026)

Towards Effective and Efficient Continual Pre-training of Large Language Models
por: Chen, Jie, et al.
Publicado: (2024)

Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
por: Zhuang, Xinlin, et al.
Publicado: (2025)

How to Set the Learning Rate for Large-Scale Pre-training?
por: Zhou, Yunhua, et al.
Publicado: (2026)

AudioKV: KV Cache Eviction in Efficient Large Audio Language Models
por: Wang, Yuxuan, et al.
Publicado: (2026)

Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration
por: Wang, Jingbo, et al.
Publicado: (2025)

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model
por: Liu, Ting, et al.
Publicado: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
por: Yan, Junbing, et al.
Publicado: (2024)

GPR: Towards a Generative Pre-trained One-Model Paradigm for Large-Scale Advertising Recommendation
por: Zhang, Jun, et al.
Publicado: (2025)

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
por: Wei, Qingyan, et al.
Publicado: (2025)

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
por: Yi, Rongjie, et al.
Publicado: (2024)

DocMamba: Efficient Document Pre-training with State Space Model
por: Hu, Pengfei, et al.
Publicado: (2024)

LA POBREZA SEGÚN EL OPUS DEI: EL OPUS DEI ENTENDIDO A TRAVÉS DE LA POBREZA
por: María Bargo
Publicado: (2018)

Efficient Transferability Assessment for Selection of Pre-trained Detectors
por: Wang, Zhao, et al.
Publicado: (2024)

ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-training
por: Jiang, Zhouqiang, et al.
Publicado: (2024)

SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching
por: Liu, Jiacheng, et al.
Publicado: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
por: Liu, Zhiyuan, et al.
Publicado: (2025)