LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914239040454656 |
|---|---|
| author | Zhu, Liya Cong, Peizhuang Ding, Jingzhe Ji, Aowei Wu, Wenya Hou, Jiani Wu, Chunjie Gao, Xiang Liu, Jingkai Huan, Zhou Sun, Xuelei Yang, Yang Jiao, Jianpeng Hu, Liang Chen, Xinjie Liu, Jiashuo Yang, Tong Wang, Zaiyuan Zhang, Ge Huang, Wenhao |
| author_facet | Zhu, Liya Cong, Peizhuang Ding, Jingzhe Ji, Aowei Wu, Wenya Hou, Jiani Wu, Chunjie Gao, Xiang Liu, Jingkai Huan, Zhou Sun, Xuelei Yang, Yang Jiao, Jianpeng Hu, Liang Chen, Xinjie Liu, Jiashuo Yang, Tong Wang, Zaiyuan Zhang, Ge Huang, Wenhao |
| contents | Large Language Models (LLMs) perform well on standard reasoning and question-answering benchmarks, yet such evaluations often fail to capture their ability to handle long-tail, expertise-intensive knowledge in real-world professional scenarios. We introduce LPFQA, a long-tail knowledge benchmark derived from authentic professional forum discussions, covering 7 academic and industrial domains with 430 curated tasks grounded in practical expertise. LPFQA evaluates specialized reasoning, domain-specific terminology understanding, and contextual interpretation, and adopts a hierarchical difficulty structure to ensure semantic clarity and uniquely identifiable answers. Experiments on over multiple mainstream LLMs reveal substantial performance gaps, particularly on tasks requiring deep domain reasoning, exposing limitations overlooked by existing benchmarks. Overall, LPFQA provides an authentic and discriminative evaluation framework that complements prior benchmarks and informs future LLM development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_06346 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluation Zhu, Liya Cong, Peizhuang Ding, Jingzhe Ji, Aowei Wu, Wenya Hou, Jiani Wu, Chunjie Gao, Xiang Liu, Jingkai Huan, Zhou Sun, Xuelei Yang, Yang Jiao, Jianpeng Hu, Liang Chen, Xinjie Liu, Jiashuo Yang, Tong Wang, Zaiyuan Zhang, Ge Huang, Wenhao Artificial Intelligence Computation and Language Large Language Models (LLMs) perform well on standard reasoning and question-answering benchmarks, yet such evaluations often fail to capture their ability to handle long-tail, expertise-intensive knowledge in real-world professional scenarios. We introduce LPFQA, a long-tail knowledge benchmark derived from authentic professional forum discussions, covering 7 academic and industrial domains with 430 curated tasks grounded in practical expertise. LPFQA evaluates specialized reasoning, domain-specific terminology understanding, and contextual interpretation, and adopts a hierarchical difficulty structure to ensure semantic clarity and uniquely identifiable answers. Experiments on over multiple mainstream LLMs reveal substantial performance gaps, particularly on tasks requiring deep domain reasoning, exposing limitations overlooked by existing benchmarks. Overall, LPFQA provides an authentic and discriminative evaluation framework that complements prior benchmarks and informs future LLM development. |
| title | LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluation |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2511.06346 |