Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915171533848576 |
|---|---|
| author | Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding |
| author_facet | Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding |
| contents | Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding Computation and Language Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation. |
| title | Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.17800 |