Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.19002 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914409342828544 |
|---|---|
| author | Łajewska, Weronika Missault, Paul Davidson, George Mansour, Saab |
| author_facet | Łajewska, Weronika Missault, Paul Davidson, George Mansour, Saab |
| contents | Simulation of surveys using LLMs is emerging as a powerful application for generating human-like responses at scale. Prior work evaluates survey simulation using metrics borrowed from other domains, which are often ad hoc, fragmented, and non-standardized, leading to results that are difficult to compare. Moreover, existing metrics focus mainly on accuracy or distributional measures, overlooking the critical dimension of ranking alignment. In practice, a simulation can achieve high accuracy while still failing to capture the option most preferred by humans - a distinction that is critical in decision-making applications. We introduce RADIUS, a comprehensive two-dimensional alignment suite for survey simulation that captures: 1) RAnking alignment and 2) DIstribUtion alignment, each complemented by statistical Significance testing. RADIUS highlights the limitations of existing metrics, enables more meaningful evaluation of survey simulation, and provides an open-source implementation for reproducible and comparable assessment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19002 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation Łajewska, Weronika Missault, Paul Davidson, George Mansour, Saab Computation and Language Simulation of surveys using LLMs is emerging as a powerful application for generating human-like responses at scale. Prior work evaluates survey simulation using metrics borrowed from other domains, which are often ad hoc, fragmented, and non-standardized, leading to results that are difficult to compare. Moreover, existing metrics focus mainly on accuracy or distributional measures, overlooking the critical dimension of ranking alignment. In practice, a simulation can achieve high accuracy while still failing to capture the option most preferred by humans - a distinction that is critical in decision-making applications. We introduce RADIUS, a comprehensive two-dimensional alignment suite for survey simulation that captures: 1) RAnking alignment and 2) DIstribUtion alignment, each complemented by statistical Significance testing. RADIUS highlights the limitations of existing metrics, enables more meaningful evaluation of survey simulation, and provides an open-source implementation for reproducible and comparable assessment. |
| title | RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.19002 |