Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.01896 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917116460924928 |
|---|---|
| author | Yu, Jinzheng Xu, Yang Li, Haozhen Li, Junqi Feng, Yifan Zhu, Ligu Shen, Hao Shi, Lei |
| author_facet | Yu, Jinzheng Xu, Yang Li, Haozhen Li, Junqi Feng, Yifan Zhu, Ligu Shen, Hao Shi, Lei |
| contents | Online Public Opinion Reports consolidate news and social media for timely crisis management by governments and enterprises. While large language models have made automated report generation technically feasible, systematic research in this specific area remains notably absent, particularly lacking formal task definitions and corresponding benchmarks. To bridge this gap, we define the Automated Online Public Opinion Report Generation (OPOR-GEN) task and construct OPOR-BENCH, an event-centric dataset covering 463 crisis events with their corresponding news articles, social media posts, and a reference summary. To evaluate report quality, we propose OPOR-EVAL, a novel agent-based framework that simulates human expert evaluation by analyzing generated reports in context. Experiments with frontier models demonstrate that our framework achieves high correlation with human judgments. Our comprehensive task definition, benchmark dataset, and evaluation framework provide a solid foundation for future research in this critical domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_01896 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OPOR-Bench: Evaluating Large Language Models on Online Public Opinion Report Generation Yu, Jinzheng Xu, Yang Li, Haozhen Li, Junqi Feng, Yifan Zhu, Ligu Shen, Hao Shi, Lei Computation and Language I.2.7 Online Public Opinion Reports consolidate news and social media for timely crisis management by governments and enterprises. While large language models have made automated report generation technically feasible, systematic research in this specific area remains notably absent, particularly lacking formal task definitions and corresponding benchmarks. To bridge this gap, we define the Automated Online Public Opinion Report Generation (OPOR-GEN) task and construct OPOR-BENCH, an event-centric dataset covering 463 crisis events with their corresponding news articles, social media posts, and a reference summary. To evaluate report quality, we propose OPOR-EVAL, a novel agent-based framework that simulates human expert evaluation by analyzing generated reports in context. Experiments with frontier models demonstrate that our framework achieves high correlation with human judgments. Our comprehensive task definition, benchmark dataset, and evaluation framework provide a solid foundation for future research in this critical domain. |
| title | OPOR-Bench: Evaluating Large Language Models on Online Public Opinion Report Generation |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2512.01896 |