Cover Image

Finetune-Informed Pretraining Boosts Downstream Performance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Faysal, Atik, Rostami, Mohammad, Roshan, Reihaneh Gh., Muralidhar, Nikhil, Wang, Huaxia
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Semi-Supervised Masked Autoencoders: Unlocking Vision Transformer Potential with Limited Data
by: Faysal, Atik, et al.
Published: (2026)

DenoMAE2.0: Improving Denoising Masked Autoencoders by Classifying Local Patches
by: Faysal, Atik, et al.
Published: (2025)

NMformer: A Transformer for Noisy Modulation Classification in Wireless Communication
by: Faysal, Atik, et al.
Published: (2024)

Large Language Models Can Perform Automatic Modulation Classification via Discretized Self-supervised Candidate Retrieval
by: Rostami, Mohammad, et al.
Published: (2025)

Plug-and-Play AMC: Context Is King in Training-Free, Open-Set Modulation with LLMs
by: Rostami, Mohammad, et al.
Published: (2025)

Meta-Task: A Method-Agnostic Framework for Learning to Regularize in Few-Shot Learning
by: Rostami, Mohammad, et al.
Published: (2024)

DenoMAE: A Multimodal Autoencoder for Denoising Modulation Signals
by: Faysal, Atik, et al.
Published: (2025)

SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning
by: Bouhsine, Taha, et al.
Published: (2024)

CageDroneRF: A Large-Scale RF Benchmark and Toolkit for Drone Perception
by: Rostami, Mohammad, et al.
Published: (2026)

On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm
by: Zhou, Zhanpeng, et al.
Published: (2024)

OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
by: Yue, Sheng, et al.
Published: (2024)

LLM-Inspired Pretrain-Then-Finetune for Small-Data, Large-Scale Optimization
by: Zhang, Zishi, et al.
Published: (2026)

Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Text
by: Jarca, Andrei, et al.
Published: (2025)

Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning
by: Banayeeanzade, Amin, et al.
Published: (2025)

The Impact of Initialization on LoRA Finetuning Dynamics
by: Hayou, Soufiane, et al.
Published: (2024)

Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
by: Wagenmaker, Andrew, et al.
Published: (2025)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
by: Liu, Yuxing, et al.
Published: (2026)

Fast and Accurate Probing of In-Training LLMs' Downstream Performances
by: Liu, Zhichen, et al.
Published: (2026)

From Basic to Extra Features: Hypergraph Transformer Pretrain-then-Finetuning for Balanced Clinical Predictions on EHR
by: Xu, Ran, et al.
Published: (2024)

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
by: Hong, Matthew M., et al.
Published: (2026)

BiSSL: Enhancing the Alignment Between Self-Supervised Pretraining and Downstream Fine-Tuning via Bilevel Optimization
by: Zakarias, Gustav Wagner, et al.
Published: (2024)

Scaling Laws for Predicting Downstream Performance in LLMs
by: Chen, Yangyi, et al.
Published: (2024)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
by: Tang, Yang, et al.
Published: (2025)

Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models
by: Hu, Zizhao, et al.
Published: (2025)

Adjusting Pretrained Backbones for Performativity
by: Demirel, Berker, et al.
Published: (2024)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

Laying Anchors: Semantically Priming Numerals in Language Modeling
by: Sharma, Mandar, et al.
Published: (2024)

Learning Dynamics of VLM Finetuning
by: Zhang, Jusheng, et al.
Published: (2025)

Boosting Relational Deep Learning with Pretrained Tabular Models
by: Lachi, Veronica, et al.
Published: (2025)

CIMGEN: Controlled Image Manipulation by Finetuning Pretrained Generative Models on Limited Data
by: Gudavalli, Chandrakanth, et al.
Published: (2024)

Learning Transferable Sensor Models via Language-Informed Pretraining
by: Chen, Yuliang, et al.
Published: (2026)

Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection
by: Livernoche, Victor, et al.
Published: (2026)

Higher-Order Transformers With Kronecker-Structured Attention
by: Omranpour, Soroush, et al.
Published: (2024)

How to Weight Multitask Finetuning? Fast Previews via Bayesian Model-Merging
by: Maldonado, Hugo Monzón, et al.
Published: (2024)

FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
by: Liu, Xiao-Yang, et al.
Published: (2024)

RefiDiff: Progressive Refinement Diffusion for Efficient Missing Data Imputation
by: Ahamed, Md Atik, et al.
Published: (2025)

Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs
by: Zhang, Yifei, et al.
Published: (2024)

Weakly Supervised Pretraining and Multi-Annotator Supervised Finetuning for Facial Wrinkle Detection
by: Moon, Ik Jun, et al.
Published: (2024)