Cover Image

Train Once, Answer All: Many Pretraining Experiments for the Cost of One

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bordt, Sebastian, Pawelczyk, Martin
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queries
by: Yan, Tianyi Lorena, et al.
Published: (2025)

How Much Can We Forget about Data Contamination?
by: Bordt, Sebastian, et al.
Published: (2024)

Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models
by: Bordt, Sebastian, et al.
Published: (2024)

Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
by: Méloux, Maxime, et al.
Published: (2025)

Many-to-English Machine Translation Tools, Data, and Pretrained Models
by: Gowda, Thamme, et al.
Published: (2021)

Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
by: Lieberum, Tom, et al.
Published: (2024)

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
by: Xiong, Zheyang, et al.
Published: (2024)

Proving that Cryptic Crossword Clue Answers are Correct
by: Andrews, Martin, et al.
Published: (2024)

One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels
by: Seshadri, Amrit Diggavi
Published: (2025)

One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMs
by: Li, Yinghui, et al.
Published: (2025)

No One Size Fits All: QueryBandits for Hallucination Mitigation
by: Cho, Nicole, et al.
Published: (2026)

One Model for All: Multi-Objective Controllable Language Models
by: He, Qiang, et al.
Published: (2026)

When Answers Stray from Questions: Hallucination Detection via Question-Answer Orthogonal Decomposition
by: Yao, Siyang, et al.
Published: (2026)

Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization
by: Li, Hanyu, et al.
Published: (2025)

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
by: You, Haoran, et al.
Published: (2024)

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
by: Li, Kenneth, et al.
Published: (2023)

Once Upon an Input: Reasoning via Per-Instance Program Synthesis
by: Stein, Adam, et al.
Published: (2025)

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
by: Tang, Yang, et al.
Published: (2025)

PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training
by: Kumar, Harsh, et al.
Published: (2026)

All You Need is One: Capsule Prompt Tuning with a Single Vector
by: Liu, Yiyang, et al.
Published: (2025)

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining
by: Fan, Dongyang, et al.
Published: (2025)

To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
by: Kothapalli, Vignesh, et al.
Published: (2025)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
by: Lu, Liming, et al.
Published: (2026)

Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment
by: Bu, Yuyan, et al.
Published: (2026)

Pretrained Hybrids with MAD Skills
by: Roberts, Nicholas, et al.
Published: (2024)

Many-Shot In-Context Learning
by: Agarwal, Rishabh, et al.
Published: (2024)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
by: Wang, Yiping, et al.
Published: (2025)

HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs
by: Chen, Junying, et al.
Published: (2023)

One Size Does Not Fit All: A Distribution-Aware Sparsification for More Precise Model Merging
by: Luo, Yingfeng, et al.
Published: (2025)

xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
by: Qian, Cheng, et al.
Published: (2025)

Answer Matching Outperforms Multiple Choice for Language Model Evaluation
by: Chandak, Nikhil, et al.
Published: (2025)

SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?
by: Kirchhof, Michael, et al.
Published: (2025)

Measuring and Reducing LLM Hallucination without Gold-Standard Answers
by: Wei, Jiaheng, et al.
Published: (2024)

Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation
by: Baan, Joris, et al.
Published: (2026)

ASAG2024: A Combined Benchmark for Short Answer Grading
by: Meyer, Gérôme, et al.
Published: (2024)

TPTT: Transforming Pretrained Transformers into Titans
by: Furfaro, Fabien
Published: (2025)

RLP: Reinforcement as a Pretraining Objective
by: Hatamizadeh, Ali, et al.
Published: (2025)

Memorization Dynamics of Fill-in-the-Middle Pretraining
by: von Arx, Tobias, et al.
Published: (2026)

All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling
by: Marconato, Emanuele, et al.
Published: (2024)

Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
by: Cao, Sheng, et al.
Published: (2025)