Cover Image

Synthetic bootstrapped pretraining

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Zitong, Zhang, Aonan, Liu, Hong, Hashimoto, Tatsunori, Candès, Emmanuel, Wang, Chong, Pang, Ruoming
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Synthetic continued pretraining
by: Yang, Zitong, et al.
Published: (2024)

Towards Execution-Grounded Automated AI Research
by: Si, Chenglei, et al.
Published: (2026)

s1: Simple test-time scaling
by: Muennighoff, Niklas, et al.
Published: (2025)

Language Models with Conformal Factuality Guarantees
by: Mohri, Christopher, et al.
Published: (2024)

Benchmarking Distributional Alignment of Large Language Models
by: Meister, Nicole, et al.
Published: (2024)

The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
by: Si, Chenglei, et al.
Published: (2025)

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
by: Si, Chenglei, et al.
Published: (2024)

Observational Scaling Laws and the Predictability of Language Model Performance
by: Ruan, Yangjun, et al.
Published: (2024)

Synthetic Data for any Differentiable Target
by: Thrush, Tristan, et al.
Published: (2026)

Linguistic Calibration of Long-Form Generations
by: Band, Neil, et al.
Published: (2024)

Removing RLHF Protections in GPT-4 via Fine-Tuning
by: Zhan, Qiusi, et al.
Published: (2023)

Reasoning to Learn from Latent Thoughts
by: Ruan, Yangjun, et al.
Published: (2025)

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
by: Dubois, Yann, et al.
Published: (2024)

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users
by: Singh, Anikait, et al.
Published: (2025)

Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?
by: Findeis, Arduin, et al.
Published: (2025)

Agentic Adversarial QA for Improving Domain-Specific LLMs
by: Grari, Vincent, et al.
Published: (2026)

Graph-based Uncertainty Metrics for Long-form Language Model Outputs
by: Jiang, Mingjian, et al.
Published: (2024)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
by: Rando, Stefano, et al.
Published: (2025)

Can Unconfident LLM Annotations Be Used for Confident Conclusions?
by: Gligorić, Kristina, et al.
Published: (2024)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
by: Liang, Wen, et al.
Published: (2024)

Do pretrained Transformers Learn In-Context by Gradient Descent?
by: Shen, Lingfeng, et al.
Published: (2023)

Event Causality Identification with Synthetic Control
by: Wang, Haoyu, et al.
Published: (2025)

Eliciting Language Model Behaviors with Investigator Agents
by: Li, Xiang Lisa, et al.
Published: (2025)

Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
by: Hu, Michael Y., et al.
Published: (2025)

Self-Improving Pretraining: using post-trained models to pretrain better models
by: Tan, Ellen Xiaoqing, et al.
Published: (2026)

ks-lit-3m: A 3.1 million word kashmiri text dataset for large language model pretraining
by: Malik, Haq Nawaz
Published: (2026)

Automated Hypothesis Validation with Agentic Sequential Falsifications
by: Huang, Kexin, et al.
Published: (2025)

Collocational bootstrapping: A hypothesis about the learning of subject-verb agreement in humans and neural networks
by: Hobbs, Claire, et al.
Published: (2026)

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
by: Dubois, Yann, et al.
Published: (2023)

Identifying the Risks of LM Agents with an LM-Emulated Sandbox
by: Ruan, Yangjun, et al.
Published: (2023)

GAUSS: Benchmarking Structured Mathematical Skills for Large Language Models
by: Zhang, Yue, et al.
Published: (2025)

TRIZ-RAGNER: A Retrieval-Augmented Large Language Model for TRIZ-Aware Named Entity Recognition in Patent-Based Contradiction Mining
by: Xu, Zitong, et al.
Published: (2026)

UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models
by: Xu, Xin, et al.
Published: (2025)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
by: Reddy, K. Sahit, et al.
Published: (2025)

A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning
by: Hong, Ruixin, et al.
Published: (2023)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
by: Sun, Yu, et al.
Published: (2024)

A Taxonomy of Prompt Defects in LLM Systems
by: Tian, Haoye, et al.
Published: (2025)

SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization
by: Mishra, Prakamya, et al.
Published: (2024)

ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
by: Lu, Jiarui, et al.
Published: (2024)

Scaling Laws of Synthetic Data for Language Models
by: Qin, Zeyu, et al.
Published: (2025)