Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911307418042368 |
|---|---|
| author | Chen, Guanyu Wang, Peiyang Jiang, Yizhou Liu, Yuqian Zhao, Chujie Fang, Ying Zhang, Tianren Chen, Feng |
| author_facet | Chen, Guanyu Wang, Peiyang Jiang, Yizhou Liu, Yuqian Zhao, Chujie Fang, Ying Zhang, Tianren Chen, Feng |
| contents | Large language models (LLMs) have been able to perform various forms of reasoning tasks in a wide range of scenarios, but are they truly engaging in task abstraction and rule-based reasoning beyond mere memorization? To answer this question, we propose a novel experimental approach, Misleading Fine-Tuning (MisFT), to examine whether LLMs perform abstract reasoning by altering their original understanding of fundamental rules. In particular, by constructing datasets with math expressions or logical formulas that contradict correct principles, we fine-tune the model to learn those contradictory rules and assess its generalization ability on unseen test domains. Through a series of experiments, we find that current LLMs are capable of applying contradictory rules to solve practical math word problems and natural language reasoning tasks, implying the presence of an internal mechanism in LLMs that abstracts before reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16401 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them Chen, Guanyu Wang, Peiyang Jiang, Yizhou Liu, Yuqian Zhao, Chujie Fang, Ying Zhang, Tianren Chen, Feng Machine Learning Large language models (LLMs) have been able to perform various forms of reasoning tasks in a wide range of scenarios, but are they truly engaging in task abstraction and rule-based reasoning beyond mere memorization? To answer this question, we propose a novel experimental approach, Misleading Fine-Tuning (MisFT), to examine whether LLMs perform abstract reasoning by altering their original understanding of fundamental rules. In particular, by constructing datasets with math expressions or logical formulas that contradict correct principles, we fine-tune the model to learn those contradictory rules and assess its generalization ability on unseen test domains. Through a series of experiments, we find that current LLMs are capable of applying contradictory rules to solve practical math word problems and natural language reasoning tasks, implying the presence of an internal mechanism in LLMs that abstracts before reasoning. |
| title | Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.16401 |