Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917160029257728 |
|---|---|
| author | Li, Wenkai Ng, Lynnette Hui Xian Liu, Andy Fried, Daniel |
| author_facet | Li, Wenkai Ng, Lynnette Hui Xian Liu, Andy Fried, Daniel |
| contents | The study of negotiation styles dates back to Aristotle's ethos-pathos-logos rhetoric. Prior efforts primarily studied the success of negotiation agents. Here, we shift the focus towards the styles of negotiation strategies. Our focus is the strategic dialogue board game Diplomacy, which affords rich natural language negotiation and measures of game success. We used LLM-as-a-judge to annotate a large human-human set of Diplomacy games for fine-grained negotiation tactics from a sociologically-grounded taxonomy. Using a combination of the It Takes Two and WebDiplomacy datasets, we demonstrate the reliability of our LLM-as-a-Judge framework and show strong correlations between negotiation features and success in the Diplomacy setting. Lastly, we investigate the differences between LLM and human negotiation strategies and show that fine-tuning can steer LLM agents toward more human-like negotiation behaviors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_18292 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy Li, Wenkai Ng, Lynnette Hui Xian Liu, Andy Fried, Daniel Computers and Society Computation and Language The study of negotiation styles dates back to Aristotle's ethos-pathos-logos rhetoric. Prior efforts primarily studied the success of negotiation agents. Here, we shift the focus towards the styles of negotiation strategies. Our focus is the strategic dialogue board game Diplomacy, which affords rich natural language negotiation and measures of game success. We used LLM-as-a-judge to annotate a large human-human set of Diplomacy games for fine-grained negotiation tactics from a sociologically-grounded taxonomy. Using a combination of the It Takes Two and WebDiplomacy datasets, we demonstrate the reliability of our LLM-as-a-Judge framework and show strong correlations between negotiation features and success in the Diplomacy setting. Lastly, we investigate the differences between LLM and human negotiation strategies and show that fine-tuning can steer LLM agents toward more human-like negotiation behaviors. |
| title | Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy |
| topic | Computers and Society Computation and Language |
| url | https://arxiv.org/abs/2512.18292 |