Cover Image

Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Szep, Marton, Rueckert, Daniel, von Eisenhart-Rothe, Rüdiger, Hinterwimmer, Florian
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
by: Szep, Marton, et al.
Published: (2026)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression
by: Mittu, Fazal, et al.
Published: (2024)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models
by: Kostikova, Aida, et al.
Published: (2025)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

Harnessing Large Language Models: Fine-tuned BERT for Detecting Charismatic Leadership Tactics in Natural Language
by: Saeid, Yasser, et al.
Published: (2024)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
by: Yu, Zhang Ze, et al.
Published: (2023)

DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
by: Huang, Wei, et al.
Published: (2025)

Efficient End-to-end Language Model Fine-tuning on Graphs
by: Xue, Rui, et al.
Published: (2023)

The Impact of Fine-tuning Large Language Models on Automated Program Repair
by: Macháček, Roman, et al.
Published: (2025)

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
by: Marín, Javier
Published: (2025)

PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning
by: Zou, Jiaru, et al.
Published: (2024)

LLM4DistReconfig: A Fine-tuned Large Language Model for Power Distribution Network Reconfiguration
by: Christou, Panayiotis, et al.
Published: (2025)

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models
by: Li, Qianxi, et al.
Published: (2023)

A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding
by: Shen, Yiqing, et al.
Published: (2024)

Disentangling Exploration of Large Language Models by Optimal Exploitation
by: Grams, Tim, et al.
Published: (2025)

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
by: Wang, Shuhe, et al.
Published: (2024)

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation
by: Huang, Tiansheng, et al.
Published: (2025)

Fine-tuning Large Language Models for Entity Matching
by: Steiner, Aaron, et al.
Published: (2024)

The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
by: Jeong, Daniel P., et al.
Published: (2024)

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
by: Xie, Juncheng, et al.
Published: (2024)

A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations
by: Laskar, Md Tahmid Rahman, et al.
Published: (2024)

Efficient Model Development through Fine-tuning Transfer
by: Lin, Pin-Jie, et al.
Published: (2025)

LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning
by: Huang, Wei, et al.
Published: (2026)

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage
by: Song, Yuda, et al.
Published: (2024)

A Study on Large Language Models' Limitations in Multiple-Choice Question Answering
by: Khatun, Aisha, et al.
Published: (2024)

A Study of Backdoors in Instruction Fine-tuned Language Models
by: Raghuram, Jayaram, et al.
Published: (2024)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
by: Chang, Aofei, et al.
Published: (2024)

G-Loss: Graph-Guided Fine-Tuning of Language Models
by: Sharma, Aditya, et al.
Published: (2026)

Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models
by: Peng, Zhiyuan, et al.
Published: (2024)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
by: Yan, Yuxuan, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies
by: Wang, Luping, et al.
Published: (2024)