Cover Image

Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Hansi, Hui, Kai, Zhuang, Honglei, Qin, Zhen, Yue, Zhenrui, Zamani, Hamed, Alon, Dana
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inference Scaling for Long-Context Retrieval Augmented Generation
by: Yue, Zhenrui, et al.
Published: (2024)

Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
by: Jin, Bowen, et al.
Published: (2025)

Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding
by: Zeng, Hansi, et al.
Published: (2024)

Scaling Sparse and Dense Retrieval in Decoder-Only LLMs
by: Zeng, Hansi, et al.
Published: (2025)

Hybrid Latent Reasoning via Reinforcement Learning
by: Yue, Zhenrui, et al.
Published: (2025)

CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
by: Zeng, Hansi, et al.
Published: (2026)

Multimodal Reranking for Knowledge-Intensive Visual Question Answering
by: Wen, Haoyang, et al.
Published: (2024)

Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Models
by: Samarinas, Chris, et al.
Published: (2024)

RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
by: Li, Mingchen, et al.
Published: (2026)

Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach
by: Zeng, Shenglai, et al.
Published: (2025)

Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models
by: Salemi, Alireza, et al.
Published: (2024)

Hypencoder: Hypernetworks for Information Retrieval
by: Killingback, Julian, et al.
Published: (2025)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
by: Zhang, Haode, et al.
Published: (2022)

Structure-aware Fine-tuning for Code Pre-trained Models
by: Wu, Jiayi, et al.
Published: (2024)

Can LLMs Predict Citation Intent? An Experimental Analysis of In-context Learning and Fine-tuning on Open LLMs
by: Koloveas, Paris, et al.
Published: (2025)

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages
by: Bahad, Sankalp, et al.
Published: (2024)

On Predicting the Post-training Potential of Pre-trained LLMs
by: Li, Xiaoyuan, et al.
Published: (2026)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem
by: Wang, Yubo, et al.
Published: (2025)

Open-Vocabulary Federated Learning with Multimodal Prototyping
by: Zeng, Huimin, et al.
Published: (2024)

A General Framework to Enhance Fine-tuning-based LLM Unlearning
by: Ren, Jie, et al.
Published: (2025)

MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarization
by: Balde, Gunjan, et al.
Published: (2024)

SPAFIT: Stratified Progressive Adaptation Fine-tuning for Pre-trained Large Language Models
by: Arora, Samir, et al.
Published: (2024)

Pre-training, Fine-tuning and Re-ranking: A Three-Stage Framework for Legal Question Answering
by: Ni, Shiwen, et al.
Published: (2024)

Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training
by: Zhang, Haode, et al.
Published: (2023)

Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
by: Zhou, Huichi, et al.
Published: (2025)

Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition
by: Yao, Zheng, et al.
Published: (2025)

Retrieval Augmented Conversational Recommendation with Reinforcement Learning
by: Yue, Zhenrui, et al.
Published: (2026)

MADLLM: Multivariate Anomaly Detection via Pre-trained LLMs
by: Tao, Wei, et al.
Published: (2025)

Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
by: Samarinas, Chris, et al.
Published: (2025)

ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classification
by: Zhu, Yaxin, et al.
Published: (2023)

Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Models
by: Salemi, Alireza, et al.
Published: (2024)

Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization
by: Salemi, Alireza, et al.
Published: (2024)

Evaluating Retrieval Quality in Retrieval-Augmented Generation
by: Salemi, Alireza, et al.
Published: (2024)

Adapters Mixup: Mixing Parameter-Efficient Adapters to Enhance the Adversarial Robustness of Fine-tuned Pre-trained Text Classifiers
by: Nguyen, Tuc, et al.
Published: (2024)

RadarPLM: Adapting Pre-trained Language Models for Marine Radar Target Detection by Selective Fine-tuning
by: Hu, Qiying, et al.
Published: (2025)

Breaking Free Transformer Models: Task-specific Context Attribution Promises Improved Generalizability Without Fine-tuning Pre-trained LLMs
by: Tytarenko, Stepan, et al.
Published: (2024)

Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!
by: Zhang, Zhexin, et al.
Published: (2025)

Fine-tuning a Large Language Model for Automating Computational Fluid Dynamics Simulations
by: Dong, Zhehao, et al.
Published: (2025)

Learning from Natural Language Feedback for Personalized Question Answering
by: Salemi, Alireza, et al.
Published: (2025)