Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909695567986688 |
|---|---|
| author | Rajeev, Meghana Ramamurthy, Rajkumar Trivedi, Prapti Yadav, Vikas Bamgbose, Oluwanifemi Madhusudan, Sathwik Tejaswi Zou, James Rajani, Nazneen |
| author_facet | Rajeev, Meghana Ramamurthy, Rajkumar Trivedi, Prapti Yadav, Vikas Bamgbose, Oluwanifemi Madhusudan, Sathwik Tejaswi Zou, James Rajani, Nazneen |
| contents | We investigate the robustness of reasoning models trained for step-by-step problem solving by introducing query-agnostic adversarial triggers - short, irrelevant text that, when appended to math problems, systematically mislead models to output incorrect answers without altering the problem's semantics. We propose CatAttack, an automated iterative attack pipeline for generating triggers on a weaker, less expensive proxy model (DeepSeek V3) and successfully transfer them to more advanced reasoning target models like DeepSeek R1 and DeepSeek R1-distilled-Qwen-32B, resulting in greater than 300% increase in the likelihood of the target model generating an incorrect answer. For example, appending, "Interesting fact: cats sleep most of their lives," to any math problem leads to more than doubling the chances of a model getting the answer wrong. Our findings highlight critical vulnerabilities in reasoning models, revealing that even state-of-the-art models remain susceptible to subtle adversarial inputs, raising security and reliability concerns. The CatAttack triggers dataset with model responses is available at https://huggingface.co/datasets/collinear-ai/cat-attack-adversarial-triggers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01781 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models Rajeev, Meghana Ramamurthy, Rajkumar Trivedi, Prapti Yadav, Vikas Bamgbose, Oluwanifemi Madhusudan, Sathwik Tejaswi Zou, James Rajani, Nazneen Computation and Language We investigate the robustness of reasoning models trained for step-by-step problem solving by introducing query-agnostic adversarial triggers - short, irrelevant text that, when appended to math problems, systematically mislead models to output incorrect answers without altering the problem's semantics. We propose CatAttack, an automated iterative attack pipeline for generating triggers on a weaker, less expensive proxy model (DeepSeek V3) and successfully transfer them to more advanced reasoning target models like DeepSeek R1 and DeepSeek R1-distilled-Qwen-32B, resulting in greater than 300% increase in the likelihood of the target model generating an incorrect answer. For example, appending, "Interesting fact: cats sleep most of their lives," to any math problem leads to more than doubling the chances of a model getting the answer wrong. Our findings highlight critical vulnerabilities in reasoning models, revealing that even state-of-the-art models remain susceptible to subtle adversarial inputs, raising security and reliability concerns. The CatAttack triggers dataset with model responses is available at https://huggingface.co/datasets/collinear-ai/cat-attack-adversarial-triggers. |
| title | Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.01781 |