Imagen de Portada

Synthetic continued pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Zitong, Band, Neil, Li, Shuangping, Candès, Emmanuel, Hashimoto, Tatsunori
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Synthetic bootstrapped pretraining
por: Yang, Zitong, et al.
Publicado: (2025)

Towards Execution-Grounded Automated AI Research
por: Si, Chenglei, et al.
Publicado: (2026)

Linguistic Calibration of Long-Form Generations
por: Band, Neil, et al.
Publicado: (2024)

Reasoning to Learn from Latent Thoughts
por: Ruan, Yangjun, et al.
Publicado: (2025)

Synthetic Data for any Differentiable Target
por: Thrush, Tristan, et al.
Publicado: (2026)

s1: Simple test-time scaling
por: Muennighoff, Niklas, et al.
Publicado: (2025)

Language Models with Conformal Factuality Guarantees
por: Mohri, Christopher, et al.
Publicado: (2024)

Observational Scaling Laws and the Predictability of Language Model Performance
por: Ruan, Yangjun, et al.
Publicado: (2024)

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
por: Si, Chenglei, et al.
Publicado: (2024)

The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
por: Si, Chenglei, et al.
Publicado: (2025)

Eliciting Language Model Behaviors with Investigator Agents
por: Li, Xiang Lisa, et al.
Publicado: (2025)

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
por: Dubois, Yann, et al.
Publicado: (2024)

Graph-based Uncertainty Metrics for Long-form Language Model Outputs
por: Jiang, Mingjian, et al.
Publicado: (2024)

Agentic Adversarial QA for Improving Domain-Specific LLMs
por: Grari, Vincent, et al.
Publicado: (2026)

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users
por: Singh, Anikait, et al.
Publicado: (2025)

Do pretrained Transformers Learn In-Context by Gradient Descent?
por: Shen, Lingfeng, et al.
Publicado: (2023)

Self-Improving Pretraining: using post-trained models to pretrain better models
por: Tan, Ellen Xiaoqing, et al.
Publicado: (2026)

Automated Hypothesis Validation with Agentic Sequential Falsifications
por: Huang, Kexin, et al.
Publicado: (2025)

Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
por: Hu, Michael Y., et al.
Publicado: (2025)

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
por: Dubois, Yann, et al.
Publicado: (2023)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
por: Reddy, K. Sahit, et al.
Publicado: (2025)

Identifying the Risks of LM Agents with an LM-Emulated Sandbox
por: Ruan, Yangjun, et al.
Publicado: (2023)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
por: Sun, Yu, et al.
Publicado: (2024)

Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
por: Ba, Yang, et al.
Publicado: (2024)

Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation?
por: Hashimoto, Wataru, et al.
Publicado: (2024)

Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks
por: Hashimoto, Wataru, et al.
Publicado: (2024)

Not All Synthetic Data Is Yours to Learn From
por: Alemohammad, Sina, et al.
Publicado: (2026)

Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework
por: Wang, Dong, et al.
Publicado: (2025)

Benchmarking Distributional Alignment of Large Language Models
por: Meister, Nicole, et al.
Publicado: (2024)

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding
por: Xu, Zhangchen, et al.
Publicado: (2025)

Synthetic Multimodal Question Generation
por: Wu, Ian, et al.
Publicado: (2024)

Evaluating Self-Supervised Learning via Risk Decomposition
por: Dubois, Yann, et al.
Publicado: (2023)

A Bitter Lesson for Data Filtering
por: Mohri, Christopher, et al.
Publicado: (2026)

CodecLM: Aligning Language Models with Tailored Synthetic Data
por: Wang, Zifeng, et al.
Publicado: (2024)

Controlled Generation for Private Synthetic Text
por: Zhao, Zihao, et al.
Publicado: (2025)

Generalizable End-to-End Tool-Use RL with Synthetic CodeGym
por: Du, Weihua, et al.
Publicado: (2025)

Shaping capabilities with token-level data filtering
por: Rathi, Neil, et al.
Publicado: (2026)

Synthetic Lyrics Detection Across Languages and Genres
por: Labrak, Yanis, et al.
Publicado: (2024)

A Synthetic Dataset for Personal Attribute Inference
por: Yukhymenko, Hanna, et al.
Publicado: (2024)

Reasoning-Driven Synthetic Data Generation and Evaluation
por: Davidson, Tim R., et al.
Publicado: (2026)