Finetune-Informed Pretraining Boosts Downstream Performance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917230180040704 |
|---|---|
| author | Faysal, Atik Rostami, Mohammad Roshan, Reihaneh Gh. Muralidhar, Nikhil Wang, Huaxia |
| author_facet | Faysal, Atik Rostami, Mohammad Roshan, Reihaneh Gh. Muralidhar, Nikhil Wang, Huaxia |
| contents | Multimodal pretraining is effective for building general-purpose representations, but in many practical deployments, only one modality is heavily used during downstream fine-tuning. Standard pretraining strategies treat all modalities uniformly, which can lead to under-optimized representations for the modality that actually matters. We propose Finetune-Informed Pretraining (FIP), a model-agnostic method that biases representation learning toward a designated target modality needed at fine-tuning time. FIP combines higher masking difficulty, stronger loss weighting, and increased decoder capacity for the target modality, without modifying the shared encoder or requiring additional supervision. When applied to masked modeling on constellation diagrams for wireless signals, FIP consistently improves downstream fine-tuned performance with no extra data or compute. FIP is simple to implement, architecture-compatible, and broadly applicable across multimodal masked modeling pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20884 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Finetune-Informed Pretraining Boosts Downstream Performance Faysal, Atik Rostami, Mohammad Roshan, Reihaneh Gh. Muralidhar, Nikhil Wang, Huaxia Machine Learning Artificial Intelligence Multimodal pretraining is effective for building general-purpose representations, but in many practical deployments, only one modality is heavily used during downstream fine-tuning. Standard pretraining strategies treat all modalities uniformly, which can lead to under-optimized representations for the modality that actually matters. We propose Finetune-Informed Pretraining (FIP), a model-agnostic method that biases representation learning toward a designated target modality needed at fine-tuning time. FIP combines higher masking difficulty, stronger loss weighting, and increased decoder capacity for the target modality, without modifying the shared encoder or requiring additional supervision. When applied to masked modeling on constellation diagrams for wireless signals, FIP consistently improves downstream fine-tuned performance with no extra data or compute. FIP is simple to implement, architecture-compatible, and broadly applicable across multimodal masked modeling pipelines. |
| title | Finetune-Informed Pretraining Boosts Downstream Performance |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2601.20884 |