Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.17800 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915171533848576 |
|---|---|
| author | Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding |
| author_facet | Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding |
| contents | Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training Yao, Yihang Cen, Zhepeng Li, Miao Han, William Zhang, Yuyou Liu, Emerson Liu, Zuxin Gan, Chuang Zhao, Ding Computation and Language Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation. |
| title | Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.17800 |