Faster Convergence for Transformer Fine-tuning with Line Search Methods

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kenneweg, Philip, Galli, Leonardo, Kenneweg, Tristan, Hammer, Barbara
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Improving Line Search Methods for Large Scale Neural Network Training
by: Kenneweg, Philip, et al.
Published: (2024)

No learning rates needed: Introducing SALSA -- Stable Armijo Line Search Adaptation
by: Kenneweg, Philip, et al.
Published: (2024)

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
by: Kenneweg, Philip, et al.
Published: (2024)

Neural Architecture Search for Sentence Classification with BERT
by: Kenneweg, Philip, et al.
Published: (2024)

Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup
by: Kenneweg, Tristan, et al.
Published: (2024)

JEPA for RL: Investigating Joint-Embedding Predictive Architectures for Reinforcement Learning
by: Kenneweg, Tristan, et al.
Published: (2025)

Uncertainty-Aware Remaining Lifespan Prediction from Images
by: Kenneweg, Tristan, et al.
Published: (2025)

Who Has The Final Say? Conformity Dynamics in ChatGPT's Selections
by: Arlinghaus, Clarissa Sabrina, et al.
Published: (2025)

Federated Loss Exploration for Improved Convergence on Non-IID Data
by: Internò, Christian, et al.
Published: (2025)

Extending Fair Null-Space Projections for Continuous Attributes to Kernel Methods
by: Störck, Felix, et al.
Published: (2025)

Towards Understanding the Influence of Training Samples on Explanations
by: Artelt, André, et al.
Published: (2024)

Fairness-Enhancing Ensemble Classification in Water Distribution Networks
by: Strotherm, Janine, et al.
Published: (2024)

Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning
by: Tong, Anh, et al.
Published: (2025)

A Minimalist Method for Fine-tuning Text-to-Image Diffusion Models
by: Miao, Yanting, et al.
Published: (2025)

On Tuning Neural ODE for Stability, Consistency and Faster Convergence
by: Akhtar, Sheikh Waqas
Published: (2023)

Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks
by: Wang, Zhili, et al.
Published: (2023)

Monte Carlo Permutation Search
by: Cazenave, Tristan
Published: (2025)

Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
by: Kim, Minseon, et al.
Published: (2025)

Calibrated Dataset Condensation for Faster Hyperparameter Search
by: Ding, Mucong, et al.
Published: (2024)

I can't see it but I can Fine-tune it: On Encrypted Fine-tuning of Transformers using Fully Homomorphic Encryption
by: Panzade, Prajwal, et al.
Published: (2024)

Continuous Fair SMOTE -- Fairness-Aware Stream Learning from Imbalanced Data
by: Lammers, Kathrin, et al.
Published: (2025)

Debiasing Sentence Embedders through Contrastive Word Pairs
by: Kenneweg, Philip, et al.
Published: (2024)

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
by: Deng, Yichuan, et al.
Published: (2024)

Unsupervised Generative Feature Transformation via Graph Contrastive Pre-training and Multi-objective Fine-tuning
by: Ying, Wangyang, et al.
Published: (2024)

Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
by: Zou, Jiaru, et al.
Published: (2025)

Speculative Coreset Selection for Task-Specific Fine-tuning
by: Zhang, Xiaoyu, et al.
Published: (2024)

Fine-tuning Timeseries Predictors Using Reinforcement Learning
by: Cazaux, Hugo, et al.
Published: (2026)

Activated LoRA: Fine-tuned LLMs for Intrinsics
by: Greenewald, Kristjan, et al.
Published: (2025)

Aggregating Low Rank Adapters in Federated Fine-tuning
by: Trautmann, Evelyn, et al.
Published: (2025)

Auto Researching, not hyperparameter tuning: Convergence Analysis of 10,000 Experiments
by: Li, Xiaoyi
Published: (2026)

Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs
by: Li, Qingyuan, et al.
Published: (2024)

Physics-Informed Graph Neural Networks for Water Distribution Systems
by: Ashraf, Inaam, et al.
Published: (2024)

The Effect of Data Poisoning on Counterfactual Explanations
by: Artelt, André, et al.
Published: (2024)

One-Class Intrusion Detection with Dynamic Graphs
by: Liuliakov, Aleksei, et al.
Published: (2025)

Skip2-LoRA: A Lightweight On-device DNN Fine-tuning Method for Low-cost Edge Devices
by: Matsutani, Hiroki, et al.
Published: (2024)

Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps
by: Yang, Ningyuan, et al.
Published: (2025)

Fine-tuning Large Language Model for Automated Algorithm Design
by: Liu, Fei, et al.
Published: (2025)

Fine-tuning Flow Matching Generative Models with Intermediate Feedback
by: Fan, Jiajun, et al.
Published: (2025)

CF-VLM:CounterFactual Vision-Language Fine-tuning
by: Zhang, Jusheng, et al.
Published: (2025)

New Insights on Unfolding and Fine-tuning Quantum Federated Learning
by: Nanayakkara, Shanika Iroshi, et al.
Published: (2025)