Cover Image

Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Junjie, Wang, Yue, Deng, Zihan, Zhu, Yan, Ma, Wenxiao, Liu, Quanying
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain
by: Yu, Junjie, et al.
Published: (2026)

Understanding Generalization from Embedding Dimension and Distributional Convergence
by: Yu, Junjie, et al.
Published: (2026)

DCHO: A Decomposition-Composition Framework for Predicting Higher-Order Brain Connectivity to Enhance Diverse Downstream Applications
by: Li, Weibin, et al.
Published: (2025)

NeuroTTT: Bridging Pretraining-Downstream Task Misalignment in EEG Foundation Models via Test-Time Training
by: Wang, Suli, et al.
Published: (2025)

Finetune-Informed Pretraining Boosts Downstream Performance
by: Faysal, Atik, et al.
Published: (2026)

Memory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agents
by: Zhang, Menglong, et al.
Published: (2024)

Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
by: Chen, Huanran, et al.
Published: (2026)

Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs
by: Yu, Xingtong, et al.
Published: (2023)

Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning
by: Qian, Fuyuan, et al.
Published: (2026)

Smoke and Mirrors in Causal Downstream Tasks
by: Cadei, Riccardo, et al.
Published: (2024)

Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding
by: Ma, Jiangyan, et al.
Published: (2023)

Vision-Language Model Selection and Reuse for Downstream Adaptation
by: Tan, Hao-Zhe, et al.
Published: (2025)

Scale-Invariance Drives Convergence in AI and Brain Representations
by: Yu, Junjie, et al.
Published: (2025)

Harvesting AlphaEarth: Benchmarking the Geospatial Foundation Model for Agricultural Downstream Tasks
by: Ma, Yuchi, et al.
Published: (2025)

A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation
by: Song, Yu, et al.
Published: (2025)

Dataset Representativeness and Downstream Task Fairness
by: Borza, Victor, et al.
Published: (2024)

Panprediction: Optimal Predictions for Any Downstream Task and Loss
by: Balakrishnan, Sivaraman, et al.
Published: (2025)

Adapting Decoder-Based Language Models for Diverse Encoder Downstream Tasks
by: Suganthan, Paul, et al.
Published: (2025)

Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks
by: Li, Miaomiao, et al.
Published: (2025)

DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks
by: Mu, Tongzhou, et al.
Published: (2024)

Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks
by: Plachouras, Christos, et al.
Published: (2025)

ECG Latent Feature Extraction with Autoencoders for Downstream Prediction Tasks
by: Harvey, Christopher, et al.
Published: (2025)

Handling Missing Data in Downstream Tasks With Distribution-Preserving Guarantees
by: Bordoloi, Rahul, et al.
Published: (2025)

Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models
by: Chijiwa, Daiki, et al.
Published: (2025)

Pretrain-Test Task Alignment Governs Generalization in In-Context Learning
by: Letey, Mary I., et al.
Published: (2025)

Attacking Attention of Foundation Models Disrupts Downstream Tasks
by: Silva, Hondamunige Prasanna, et al.
Published: (2025)

Fairness-Optimized Synthetic EHR Generation for Arbitrary Downstream Predictive Tasks
by: Tarek, Mirza Farhan Bin, et al.
Published: (2024)

Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Tasks
by: Chu, Yun-Wei, et al.
Published: (2024)

Task-tailored Pre-processing: Fair Downstream Supervised Learning
by: Sohn, Jinwon, et al.
Published: (2026)

Adversarial Attacks on Downstream Weather Forecasting Models: Application to Tropical Cyclone Trajectory Prediction
by: Deng, Yue, et al.
Published: (2025)

Task Priors: Enhancing Model Evaluation by Considering the Entire Space of Downstream Tasks
by: Patel, Niket, et al.
Published: (2025)

DSBD: Dual-Aligned Structural Basis Distillation for Graph Domain Adaptation
by: Wang, Yingxu, et al.
Published: (2026)

TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining
by: Zhang, Ruiyi, et al.
Published: (2024)

Salutary Labeling with Zero Human Annotation
by: Xiao, Wenxiao, et al.
Published: (2024)

R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models
by: Yin, Tianyi, et al.
Published: (2025)

Meta-Learned Basis Adaptation for Parametric Linear PDEs
by: Dwivedi, Vikas, et al.
Published: (2026)

VocabTailor: Dynamic Vocabulary Selection for Downstream Tasks in Small Language Models
by: Zhang, Hanling, et al.
Published: (2025)

Mixing Configurations for Downstream Prediction
by: Wang, Juntang, et al.
Published: (2025)

USBD: Universal Structural Basis Distillation for Source-Free Graph Domain Adaptation
by: Wang, Yingxu, et al.
Published: (2026)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
by: Lourie, Nicholas, et al.
Published: (2025)