Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911215408644096 |
|---|---|
| author | Peng, Xujun Kumar, Anoop Wu, Jingyu Glenn, Parker Liu, Daben |
| author_facet | Peng, Xujun Kumar, Anoop Wu, Jingyu Glenn, Parker Liu, Daben |
| contents | Retrieval-Augmented Generation (RAG) systems leverage Large Language Models (LLMs) to generate accurate and reliable responses that are grounded in retrieved context. However, LLMs often generate inconsistent outputs for semantically equivalent inputs, a problem compounded by the scarcity of consistency-focused training data and the limitations of current fine-tuning techniques in enhancing output consistency. We propose a new approach combining systematic synthetic data generation, triplet loss for better embeddings, and a novel layer-wise model merging approach. Using consistency-aware weights derived from intermediate layer activations, our method effectively integrates knowledge from specialized models. Experimental results how that our merged model significantly enhances output consistency, achieving a ~47.5\% improvement in response similarity over the baseline, thus offering a practical solution for increasing the reliability of an industrial RAG system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14915 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation Peng, Xujun Kumar, Anoop Wu, Jingyu Glenn, Parker Liu, Daben Computation and Language Retrieval-Augmented Generation (RAG) systems leverage Large Language Models (LLMs) to generate accurate and reliable responses that are grounded in retrieved context. However, LLMs often generate inconsistent outputs for semantically equivalent inputs, a problem compounded by the scarcity of consistency-focused training data and the limitations of current fine-tuning techniques in enhancing output consistency. We propose a new approach combining systematic synthetic data generation, triplet loss for better embeddings, and a novel layer-wise model merging approach. Using consistency-aware weights derived from intermediate layer activations, our method effectively integrates knowledge from specialized models. Experimental results how that our merged model significantly enhances output consistency, achieving a ~47.5\% improvement in response similarity over the baseline, thus offering a practical solution for increasing the reliability of an industrial RAG system. |
| title | Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.14915 |