Copertina

Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Huang, Chengyu, Chou, Sheng-Yen, Zhang, Zhengxin, Cardie, Claire
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization
di: Huang, Chengyu, et al.
Pubblicazione: (2025)

Better LLM Reasoning via Dual-Play
di: Zhang, Zhengxin, et al.
Pubblicazione: (2025)

Token-weighted Direct Preference Optimization with Attention
di: Huang, Chengyu, et al.
Pubblicazione: (2026)

RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
di: Jiang, Haoxiang, et al.
Pubblicazione: (2026)

Thinking Augmented Pre-training
di: Wang, Liang, et al.
Pubblicazione: (2025)

BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation
di: Wang, Chen, et al.
Pubblicazione: (2024)

Transferable Post-training via Inverse Value Learning
di: Lu, Xinyu, et al.
Pubblicazione: (2024)

How Far Are We From True Auto-Research?
di: Zhang, Zhengxin, et al.
Pubblicazione: (2026)

On Predicting the Post-training Potential of Pre-trained LLMs
di: Li, Xiaoyuan, et al.
Pubblicazione: (2026)

Variance Control via Weight Rescaling in LLM Pre-training
di: Owen, Louis, et al.
Pubblicazione: (2025)

Post-training an LLM for RAG? Train on Self-Generated Demonstrations
di: Finlayson, Matthew, et al.
Pubblicazione: (2025)

BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing
di: Wang, Chen, et al.
Pubblicazione: (2023)

Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning
di: Ye, Zhiling, et al.
Pubblicazione: (2025)

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
di: Nakamura, Taishi, et al.
Pubblicazione: (2024)

Post-training for Efficient Communication via Convention Formation
di: Hua, Yilun, et al.
Pubblicazione: (2025)

MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector
di: Fu, Wenjie, et al.
Pubblicazione: (2024)

Improving Low-Resource Knowledge Tracing Tasks by Supervised Pre-training and Importance Mechanism Fine-tuning
di: Zhang, Hengyuan, et al.
Pubblicazione: (2024)

TiMix: Text-aware Image Mixing for Effective Vision-Language Pre-training
di: Jiang, Chaoya, et al.
Pubblicazione: (2023)

Memory-Efficient Training for Text-Dependent SV with Independent Pre-trained Models
di: Farokh, Seyed Ali, et al.
Pubblicazione: (2024)

Model Merging in Pre-training of Large Language Models
di: Li, Yunshui, et al.
Pubblicazione: (2025)

TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
di: Cao, Defu, et al.
Pubblicazione: (2023)

Structural Pruning of Pre-trained Language Models via Neural Architecture Search
di: Klein, Aaron, et al.
Pubblicazione: (2024)

A Comparative Study of Pre-training and Self-training
di: Wang, Yiheng, et al.
Pubblicazione: (2024)

Data-efficient Performance Modeling via Pre-training
di: Liu, Chunting, et al.
Pubblicazione: (2025)

Prioritized Replay for RL Post-training
di: Fatemi, Mehdi
Pubblicazione: (2026)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
di: Yan, Junbing, et al.
Pubblicazione: (2024)

Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens
di: Zhang, Anqi, et al.
Pubblicazione: (2024)

DEPT: Decoupled Embeddings for Pre-training Language Models
di: Iacob, Alex, et al.
Pubblicazione: (2024)

EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation
di: Guan, Xin, et al.
Pubblicazione: (2026)

Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
di: Diao, Shizhe, et al.
Pubblicazione: (2025)

Chinese MentalBERT: Domain-Adaptive Pre-training on Social Media for Chinese Mental Health Text Analysis
di: Zhai, Wei, et al.
Pubblicazione: (2024)

nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales
di: Yao, Yiqun, et al.
Pubblicazione: (2023)

DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training
di: Wang, Zhenting, et al.
Pubblicazione: (2025)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
di: Zhang, Ying, et al.
Pubblicazione: (2024)

HLAT: High-quality Large Language Model Pre-trained on AWS Trainium
di: Fan, Haozheng, et al.
Pubblicazione: (2024)

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
di: Su, Jinyan, et al.
Pubblicazione: (2026)

Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards
di: Su, Jinyan, et al.
Pubblicazione: (2025)

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation
di: Zhang, Yuhui, et al.
Pubblicazione: (2023)

Efficient Continual Pre-training of LLMs for Low-resource Languages
di: Nag, Arijit, et al.
Pubblicazione: (2024)

Making Pre-trained Language Models Great on Tabular Prediction
di: Yan, Jiahuan, et al.
Pubblicazione: (2024)