Cover Image

Should I try multiple optimizers when fine-tuning pre-trained Transformers for NLP tasks? Should I tune their hyperparameters?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gkouti, Nefeli, Malakasiotis, Prodromos, Toumpis, Stavros, Androutsopoulos, Ion
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Replaying pre-training data improves fine-tuning
by: Kotha, Suhas, et al.
Published: (2026)

GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek
by: Loukas, Lefteris, et al.
Published: (2024)

Rethinking harmless refusals when fine-tuning foundation models
by: Pop, Florin, et al.
Published: (2024)

AUEB-Archimedes at RIRAG-2025: Is obligation concatenation really all you need?
by: Chasandras, Ioannis, et al.
Published: (2024)

Archimedes-AUEB at SemEval-2024 Task 5: LLM explains Civil Procedure
by: Chlapanis, Odysseas S., et al.
Published: (2024)

Multi-task retriever fine-tuning for domain-specific and efficient RAG
by: Béchard, Patrice, et al.
Published: (2025)

Comparing Data Augmentation Methods for End-to-End Task-Oriented Dialog Systems
by: Vlachos, Christos, et al.
Published: (2024)

LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights
by: Chlapanis, Odysseas S., et al.
Published: (2024)

Complexity-aware fine-tuning
by: Goncharov, Andrey, et al.
Published: (2025)

Response-free item difficulty modelling for multiple-choice items with fine-tuned transformers: Component-wise representation and multi-task learning
by: Netík, Jan, et al.
Published: (2026)

Immunization against harmful fine-tuning attacks
by: Rosati, Domenic, et al.
Published: (2024)

Temporal fine-tuning for early risk detection
by: Thompson, Horacio, et al.
Published: (2025)

GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations
by: Chlapanis, Odysseas S., et al.
Published: (2025)

A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks
by: Zhang, Tianyi, et al.
Published: (2024)

Combining Denoising Autoencoders with Contrastive Learning to fine-tune Transformer Models
by: Lopez-Avila, Alejo, et al.
Published: (2024)

Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitation
by: Tsirmpas, Dimitris, et al.
Published: (2025)

Labeling supervised fine-tuning data with the scaling law
by: Kong, Huanjun
Published: (2024)

Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
by: Wongkamjan, Wichayaporn, et al.
Published: (2025)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
by: Wang, Yibo, et al.
Published: (2025)

Retention analysis of edited knowledge after fine-tuning
by: Wen, Fufang, et al.
Published: (2025)

Understanding the effects of language-specific class imbalance in multilingual fine-tuning
by: Jung, Vincent, et al.
Published: (2024)

Improving LLM-based Ontology Matching with fine-tuning on synthetic data
by: Sousa, Guilherme, et al.
Published: (2025)

Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
by: Zhong, Ziqian, et al.
Published: (2025)

Topic mining based on fine-tuning Sentence-BERT and LDA
by: Li, Jianheng, et al.
Published: (2025)

A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understanding
by: Laperrière, Gaëlle, et al.
Published: (2024)

Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation
by: Ki, Dayeon, et al.
Published: (2025)

Birbal: An efficient 7B instruct-model fine-tuned with curated datasets
by: Jindal, Ashvini Kumar, et al.
Published: (2024)

Knowledge AI: Fine-tuning NLP Models for Facilitating Scientific Knowledge Extraction and Understanding
by: Muralidharan, Balaji, et al.
Published: (2024)

A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning
by: Kaliosis, Panagiotis, et al.
Published: (2024)

You can remove GPT2's LayerNorm by fine-tuning
by: Heimersheim, Stefan
Published: (2024)

LayerNorm: A key component in parameter-efficient fine-tuning
by: ValizadehAslani, Taha, et al.
Published: (2024)

In Tree Structure Should Sentence Be Generated
by: Li, Yaguang, et al.
Published: (2024)

Authors Should Label Their Own Documents
by: Ma, Marcus, et al.
Published: (2025)

Deep literature reviews: an application of fine-tuned language models to migration research
by: Iacus, Stefano M., et al.
Published: (2025)

What explains the success of cross-modal fine-tuning with ORCA?
by: García-de-Herreros, Paloma, et al.
Published: (2024)

Where Should I Study? Biased Language Models Decide! Evaluating Fairness in LMs for Academic Recommendations
by: Shailya, Krithi, et al.
Published: (2025)

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages
by: Bahad, Sankalp, et al.
Published: (2024)

Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?
by: Zeng, Hansi, et al.
Published: (2025)

The representation landscape of few-shot learning and fine-tuning in large language models
by: Doimo, Diego, et al.
Published: (2024)

How does fine-tuning improve sensorimotor representations in large language models?
by: Wu, Minghua, et al.
Published: (2026)