Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2603.10012 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911503910699008 |
|---|---|
| author | FitzGerald, Jack Bates, Dylan Lazaridis, Aristotelis Sharma, Aman Lu, Vincent King, Brian Azami, Yousif Bailey, Sean Cao, Jeremy Damianov, Peter de Haan, Kevin Madigan, Joseph McLaurin, Jeremy Kerbs, Luke Tainer, Jonathan Anderson, Dave Beck, Jonathan Cuticello, Jamie Malkerson, Colton Saltsman, Tyler |
| author_facet | FitzGerald, Jack Bates, Dylan Lazaridis, Aristotelis Sharma, Aman Lu, Vincent King, Brian Azami, Yousif Bailey, Sean Cao, Jeremy Damianov, Peter de Haan, Kevin Madigan, Joseph McLaurin, Jeremy Kerbs, Luke Tainer, Jonathan Anderson, Dave Beck, Jonathan Cuticello, Jamie Malkerson, Colton Saltsman, Tyler |
| contents | Military Large Language Models (LLMs) must provide accurate information to the warfighter in time-critical and dangerous situations. However, today's LLMs are imbued with safety behaviors that cause the LLM to refuse many legitimate queries in the military domain, particularly those related to violence, terrorism, or military technology. Our gold benchmark for assessing refusal rates, which was developed by veterans of the US Army and special forces, is to our knowledge the first dataset of its kind. We present results for refusal and deflection rates on 31 public models and 3 military models. We observe hard rejection rates as high as 98.2% and soft deflection rates ranging from 0% to 21.3%. We also present results on two additional synthetic datasets and show their correlations with the gold dataset. Finally, we perform abliteration using the Heretic library on a military-tuned gpt-oss-20b model, showing an absolute increase in answer rate of 66.5 points but an average relative decrease of 2% on other military tasks. In our concluding remarks, we argue for deeper specialization, including with mid-training and end-to-end post-training, to achieve zero refusals and maximum military task accuracy for closed military models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10012 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Measuring and Eliminating Refusals in Military Large Language Models FitzGerald, Jack Bates, Dylan Lazaridis, Aristotelis Sharma, Aman Lu, Vincent King, Brian Azami, Yousif Bailey, Sean Cao, Jeremy Damianov, Peter de Haan, Kevin Madigan, Joseph McLaurin, Jeremy Kerbs, Luke Tainer, Jonathan Anderson, Dave Beck, Jonathan Cuticello, Jamie Malkerson, Colton Saltsman, Tyler Computation and Language Artificial Intelligence Military Large Language Models (LLMs) must provide accurate information to the warfighter in time-critical and dangerous situations. However, today's LLMs are imbued with safety behaviors that cause the LLM to refuse many legitimate queries in the military domain, particularly those related to violence, terrorism, or military technology. Our gold benchmark for assessing refusal rates, which was developed by veterans of the US Army and special forces, is to our knowledge the first dataset of its kind. We present results for refusal and deflection rates on 31 public models and 3 military models. We observe hard rejection rates as high as 98.2% and soft deflection rates ranging from 0% to 21.3%. We also present results on two additional synthetic datasets and show their correlations with the gold dataset. Finally, we perform abliteration using the Heretic library on a military-tuned gpt-oss-20b model, showing an absolute increase in answer rate of 66.5 points but an average relative decrease of 2% on other military tasks. In our concluding remarks, we argue for deeper specialization, including with mid-training and end-to-end post-training, to achieve zero refusals and maximum military task accuracy for closed military models. |
| title | Measuring and Eliminating Refusals in Military Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2603.10012 |