Copertina

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Shaobo, Ouyang, Xuan, Xu, Tianyi, Hu, Yuzheng, Liu, Jialin, Chen, Guo, Zhang, Tianyu, Zheng, Junhao, Yang, Kexin, Ren, Xingzhang, Liu, Dayiheng, Zhang, Linfeng
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
di: Wang, Shaobo, et al.
Pubblicazione: (2026)

An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
di: Tian, Yuxin, et al.
Pubblicazione: (2024)

Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
di: Zhou, Yufa, et al.
Pubblicazione: (2025)

Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
di: Wang, Shaobo, et al.
Pubblicazione: (2025)

DataMan: Data Manager for Pre-training Large Language Models
di: Peng, Ru, et al.
Pubblicazione: (2025)

RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
di: Bi, Baolong, et al.
Pubblicazione: (2025)

On Predicting the Post-training Potential of Pre-trained LLMs
di: Li, Xiaoyuan, et al.
Pubblicazione: (2026)

Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
di: Wang, Shaobo, et al.
Pubblicazione: (2025)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
di: Wang, Shaobo, et al.
Pubblicazione: (2026)

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
di: Yang, An, et al.
Pubblicazione: (2024)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
di: Tang, Shengkun, et al.
Pubblicazione: (2026)

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
di: Zhou, Jiecheng, et al.
Pubblicazione: (2025)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
di: Zhou, Fan, et al.
Pubblicazione: (2024)

Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis
di: Jiao, Zhengbo, et al.
Pubblicazione: (2026)

Towards Principled Dataset Distillation: A Spectral Distribution Perspective
di: Wu, Ruixi, et al.
Pubblicazione: (2026)

EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision
di: Xu, Tianyi, et al.
Pubblicazione: (2026)

Towards Efficient LLMs Annealing with Principled Sample Selection
di: Xu, Yuanjian, et al.
Pubblicazione: (2026)

How to Set the Batch Size for Large-Scale Pre-training?
di: Zhou, Yunhua, et al.
Pubblicazione: (2026)

OPUS: Occupancy Prediction Using a Sparse Set
di: Wang, Jiabao, et al.
Pubblicazione: (2024)

GraphGPT: Generative Pre-trained Graph Eulerian Transformer
di: Zhao, Qifang, et al.
Pubblicazione: (2023)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
di: Qian, Chen, et al.
Pubblicazione: (2024)

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
di: Ren, Xuan, et al.
Pubblicazione: (2025)

Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
di: He, Yu, et al.
Pubblicazione: (2025)

Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling
di: Chen, Yujie, et al.
Pubblicazione: (2026)

Towards Effective and Efficient Continual Pre-training of Large Language Models
di: Chen, Jie, et al.
Pubblicazione: (2024)

Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
di: Zhuang, Xinlin, et al.
Pubblicazione: (2025)

How to Set the Learning Rate for Large-Scale Pre-training?
di: Zhou, Yunhua, et al.
Pubblicazione: (2026)

AudioKV: KV Cache Eviction in Efficient Large Audio Language Models
di: Wang, Yuxuan, et al.
Pubblicazione: (2026)

Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration
di: Wang, Jingbo, et al.
Pubblicazione: (2025)

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model
di: Liu, Ting, et al.
Pubblicazione: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
di: Yan, Junbing, et al.
Pubblicazione: (2024)

GPR: Towards a Generative Pre-trained One-Model Paradigm for Large-Scale Advertising Recommendation
di: Zhang, Jun, et al.
Pubblicazione: (2025)

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
di: Wei, Qingyan, et al.
Pubblicazione: (2025)

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
di: Yi, Rongjie, et al.
Pubblicazione: (2024)

DocMamba: Efficient Document Pre-training with State Space Model
di: Hu, Pengfei, et al.
Pubblicazione: (2024)

LA POBREZA SEGÚN EL OPUS DEI: EL OPUS DEI ENTENDIDO A TRAVÉS DE LA POBREZA
di: María Bargo
Pubblicazione: (2018)

Efficient Transferability Assessment for Selection of Pre-trained Detectors
di: Wang, Zhao, et al.
Pubblicazione: (2024)

ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-training
di: Jiang, Zhouqiang, et al.
Pubblicazione: (2024)

SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching
di: Liu, Jiacheng, et al.
Pubblicazione: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
di: Liu, Zhiyuan, et al.
Pubblicazione: (2025)