Copertina

Matched-Learning-Rate Analysis of Attention Drift and Transfer Retention in Fine-Tuned CLIP

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Xia, Ruize
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
di: Ran, Yide, et al.
Pubblicazione: (2025)

Gradual Fine-Tuning for Flow Matching Models
di: Thorkelsdottir, Gudrun, et al.
Pubblicazione: (2026)

Tilt Matching for Scalable Sampling and Fine-Tuning
di: Potaptchik, Peter, et al.
Pubblicazione: (2025)

Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning
di: Peleg, Amit, et al.
Pubblicazione: (2025)

Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates
di: Prashant, Parjanya Prajakta, et al.
Pubblicazione: (2026)

Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation
di: Wang, Yizheng, et al.
Pubblicazione: (2025)

Control Theoretic Approach to Fine-Tuning and Transfer Learning
di: Bayram, Erkan, et al.
Pubblicazione: (2024)

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
di: Lee, Chungpa, et al.
Pubblicazione: (2026)

Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
di: Abdi, Hossein, et al.
Pubblicazione: (2025)

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers
di: Daga, Miit, et al.
Pubblicazione: (2026)

Drift Flow Matching
di: Ma, Chenrui, et al.
Pubblicazione: (2026)

Beyond Match Maximization and Fairness: Retention-Optimized Two-Sided Matching
di: Kishimoto, Ren, et al.
Pubblicazione: (2026)

Objective Matters: Fine-Tuning Objectives Shape Safety, Robustness, and Persona Drift
di: Vennemeyer, Daniel, et al.
Pubblicazione: (2026)

Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP
di: Chen, Zixiang, et al.
Pubblicazione: (2023)

Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models
di: Jelassi, Samy, et al.
Pubblicazione: (2026)

Fine-Tuning Flow Matching via Maximum Likelihood Estimation of Reconstructions
di: Li, Zhaoyi, et al.
Pubblicazione: (2025)

Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization
di: Yao, Xinhao, et al.
Pubblicazione: (2024)

Mitigating Data Exfiltration Attacks through Layer-Wise Learning Rate Decay Fine-Tuning
di: Thellier, Elie, et al.
Pubblicazione: (2025)

Attention Drift: What Autoregressive Speculative Decoding Models Learn
di: Eldenk, Doğaç, et al.
Pubblicazione: (2026)

Contrast-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment
di: Lv, Song-Lin, et al.
Pubblicazione: (2025)

Open-Vocabulary Calibration for Fine-tuned CLIP
di: Wang, Shuoyuan, et al.
Pubblicazione: (2024)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
di: Yano, Kazuki, et al.
Pubblicazione: (2026)

TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models
di: Guan, Yushi, et al.
Pubblicazione: (2026)

Tuning Learning Rates with the Cumulative-Learning Constant
di: Faraj, Nathan
Pubblicazione: (2025)

Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic
di: Jin, Ruochen, et al.
Pubblicazione: (2024)

Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
di: Lyu, Mingyang, et al.
Pubblicazione: (2025)

The Magic Correlations: Understanding Knowledge Transfer from Pretraining to Supervised Fine-Tuning
di: Fan, Simin, et al.
Pubblicazione: (2026)

MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental Learning
di: Nicolas, Julien, et al.
Pubblicazione: (2023)

FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics
di: Du, Yupei, et al.
Pubblicazione: (2023)

DiffCLIP: Differential Attention Meets CLIP
di: Hammoud, Hasan Abed Al Kader, et al.
Pubblicazione: (2025)

Boosting Deep Ensembles with Learning Rate Tuning
di: Jin, Hongpeng, et al.
Pubblicazione: (2024)

Transfer Learning Through Conditional Quantile Matching
di: Zhang, Yikun, et al.
Pubblicazione: (2026)

SOPHON: Non-Fine-Tunable Learning to Restrain Task Transferability For Pre-trained Models
di: Deng, Jiangyi, et al.
Pubblicazione: (2024)

Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning
di: Menghini, Cristina, et al.
Pubblicazione: (2023)

Distributional Drift Detection in Medical Imaging with Sketching and Fine-Tuned Transformer
di: Wu, Yusen, et al.
Pubblicazione: (2024)

Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning
di: Tian, Zichen, et al.
Pubblicazione: (2026)

Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
di: Turan, Erkan, et al.
Pubblicazione: (2026)

Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems
di: Tauberschmidt, Jan, et al.
Pubblicazione: (2025)

SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models
di: He, Jinghan, et al.
Pubblicazione: (2024)

An Empirical Study of $μ$P Learning Rate Transfer
di: Lingle, Lucas
Pubblicazione: (2024)