Cover Image

Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Dongyue, Zhang, Ziniu, Wang, Lu, Zhang, Hongyang R.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach
by: Li, Dongyue, et al.
Published: (2024)

Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation
by: Zhang, Ziniu, et al.
Published: (2025)

Improved Regularization and Robustness for Fine-tuning in Neural Networks
by: Li, Dongyue, et al.
Published: (2021)

SEE: Continual Fine-tuning with Sequential Ensemble of Experts
by: Wang, Zhilin, et al.
Published: (2025)

Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
by: Tayaranian, Mohammadreza, et al.
Published: (2024)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
by: Yu, Zhang Ze, et al.
Published: (2023)

Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
by: Zhou, Huichi, et al.
Published: (2025)

Efficient End-to-end Language Model Fine-tuning on Graphs
by: Xue, Rui, et al.
Published: (2023)

MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Models
by: Ai, Mengting, et al.
Published: (2023)

Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
by: Xia, Wenhan, et al.
Published: (2024)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding
by: Shen, Yiqing, et al.
Published: (2024)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
by: Wang, Shuhe, et al.
Published: (2024)

Fine-tuning Large Language Models for Domain-specific Machine Translation
by: Zheng, Jiawei, et al.
Published: (2024)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
by: Zeng, Hanqing, et al.
Published: (2025)

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
by: Zhang, Renrui, et al.
Published: (2023)

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
by: Li, Yuhui, et al.
Published: (2024)

BWArea Model: Learning World Model, Inverse Dynamics, and Policy for Controllable Language Generation
by: Jia, Chengxing, et al.
Published: (2024)

Efficient Estimation of Kernel Surrogate Models for Task Attribution
by: Zhang, Zhenshuo, et al.
Published: (2026)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
by: Xu, Jing, et al.
Published: (2024)

Efficient Model Development through Fine-tuning Transfer
by: Lin, Pin-Jie, et al.
Published: (2025)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models
by: Najafi, Saeed, et al.
Published: (2024)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
by: Yao, Kai, et al.
Published: (2024)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

Advancing Parameter Efficiency in Fine-tuning via Representation Editing
by: Wu, Muling, et al.
Published: (2024)

Relational Knowledge Distillation Using Fine-tuned Function Vectors
by: Kang, Andrea, et al.
Published: (2026)

Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning
by: Krishna, Shambhavi, et al.
Published: (2025)

Whispering in Amharic: Fine-tuning Whisper for Low-resource Language
by: Gete, Dawit Ketema, et al.
Published: (2025)

Controlling Large Language Model with Latent Actions
by: Jia, Chengxing, et al.
Published: (2025)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective
by: Ghosh, Bishwamittra, et al.
Published: (2026)

PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning
by: Zou, Jiaru, et al.
Published: (2024)

Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Generator-Validator Fine-tuning
by: Xing, Junjie, et al.
Published: (2024)

Topic Modeling with Fine-tuning LLMs and Bag of Sentences
by: Schneider, Johannes
Published: (2024)

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models
by: Li, Qianxi, et al.
Published: (2023)