Cover Image

Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Glocker, Kevin, Kukk, Kätriin, Oji, Romina, Bollmann, Marcel, Kuhlmann, Marco, Kunz, Jenny
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapters
by: Oji, Romina, et al.
Published: (2025)

Properties and Challenges of LLM-Generated Explanations
by: Kunz, Jenny, et al.
Published: (2024)

What Matters for Model Merging at Scale?
by: Yadav, Prateek, et al.
Published: (2024)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
by: Kallini, Julie, et al.
Published: (2024)

Arcee's MergeKit: A Toolkit for Merging Large Language Models
by: Goddard, Charles, et al.
Published: (2024)

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models
by: Shenaj, Donald, et al.
Published: (2025)

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
by: Kim, Dahyun, et al.
Published: (2023)

LLaDA2.0: Scaling Up Diffusion Language Models to 100B
by: Bie, Tiwei, et al.
Published: (2025)

Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models
by: Puerto, Haritz, et al.
Published: (2024)

Fisher Mask Nodes for Language Model Merging
by: K, Thennal D, et al.
Published: (2024)

Scaling Inference-Efficient Language Models
by: Bian, Song, et al.
Published: (2025)

LESA: Learnable LLM Layer Scaling-Up
by: Yang, Yifei, et al.
Published: (2025)

SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation
by: Kapusuzoglu, Berkcan, et al.
Published: (2025)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Channel Merging: Preserving Specialization for Merged Experts
by: Zhang, Mingyang, et al.
Published: (2024)

Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarking
by: Acikgoz, Emre Can, et al.
Published: (2024)

Merge to Mix: Mixing Datasets via Model Merging
by: Tao, Zhixu Silvia, et al.
Published: (2025)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
by: Kumar, Sachin
Published: (2026)

Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models
by: Zeng, Linda, et al.
Published: (2026)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
by: Lu, Zhenyi, et al.
Published: (2024)

Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging
by: Wang, Zhixiang, et al.
Published: (2025)

Crafting Efficient Fine-Tuning Strategies for Large Language Models
by: Oliver, Michael, et al.
Published: (2024)

Data-driven Clustering and Merging of Adapters for On-device Large Language Models
by: Bohdal, Ondrej, et al.
Published: (2026)

Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
by: Saha, Swarnadeep, et al.
Published: (2023)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
by: Siriwardhana, Shamane, et al.
Published: (2024)

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
by: Li, Pingzhi, et al.
Published: (2023)

Merge-of-Thought Distillation
by: Shen, Zhanming, et al.
Published: (2025)

Dynamic Strategy Planning for Efficient Question Answering with Large Language Models
by: Parekh, Tanmay, et al.
Published: (2024)

LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
by: Elesedy, Hayder, et al.
Published: (2024)

Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
by: Gauthier-Caron, Thomas, et al.
Published: (2024)

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
by: Liu, Mingjie, et al.
Published: (2025)

Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks
by: Zhang, Edwin, et al.
Published: (2022)

PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
by: Goffinet, Etienne, et al.
Published: (2025)

Model Merging for Knowledge Editing
by: Fu, Zichuan, et al.
Published: (2025)

ROSA: Random Subspace Adaptation for Efficient Fine-Tuning
by: Hameed, Marawan Gamal Abdel, et al.
Published: (2024)

Dynamic Model Merging Made Slim
by: Du, Guodong, et al.
Published: (2026)

FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
by: Ma, Liqun, et al.
Published: (2024)

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability
by: Liao, Baohao, et al.
Published: (2024)

DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing
by: Ma, Xinyu, et al.
Published: (2025)