Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.01825 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917183581323264 |
|---|---|
| author | Cui, Yaxin Zeng, Yuanqiang Yan, Jiapeng Lin, Keling Ji, Kai Zeng, Jianhui Zhang, Sheng Luo, Xin Su, Binzhu Shen, Chaolai Yu, Jiahao |
| author_facet | Cui, Yaxin Zeng, Yuanqiang Yan, Jiapeng Lin, Keling Ji, Kai Zeng, Jianhui Zhang, Sheng Luo, Xin Su, Binzhu Shen, Chaolai Yu, Jiahao |
| contents | Large Language Models (LLMs) have achieved remarkable success in general benchmarks, yet their competence in commodity supply chains (CSCs) -- a domain governed by institutional rule systems and feasibility constraints -- remains under-explored. CSC decisions are shaped jointly by process stages (e.g., planning, procurement, delivery), variety-specific rules (e.g., contract specifications and delivery grades), and reasoning depth (from retrieval to multi-step analysis and decision selection). We introduce CSCBench, a 2.3K+ single-choice benchmark for CSC reasoning, instantiated through our PVC 3D Evaluation Framework (Process, Variety, and Cognition). The Process axis aligns tasks with SCOR+Enable; the Variety axis operationalizes commodity-specific rule systems under coupled material-information-financial constraints, grounded in authoritative exchange guidebooks/rulebooks and industry reports; and the Cognition axis follows Bloom's revised taxonomy. Evaluating representative LLMs under a direct prompting setting, we observe strong performance on the Process and Cognition axes but substantial degradation on the Variety axis, especially on Freight Agreements. CSCBench provides a diagnostic yardstick for measuring and improving LLM capabilities in this high-stakes domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_01825 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CSCBench: A PVC Diagnostic Benchmark for Commodity Supply Chain Reasoning Cui, Yaxin Zeng, Yuanqiang Yan, Jiapeng Lin, Keling Ji, Kai Zeng, Jianhui Zhang, Sheng Luo, Xin Su, Binzhu Shen, Chaolai Yu, Jiahao Computation and Language Large Language Models (LLMs) have achieved remarkable success in general benchmarks, yet their competence in commodity supply chains (CSCs) -- a domain governed by institutional rule systems and feasibility constraints -- remains under-explored. CSC decisions are shaped jointly by process stages (e.g., planning, procurement, delivery), variety-specific rules (e.g., contract specifications and delivery grades), and reasoning depth (from retrieval to multi-step analysis and decision selection). We introduce CSCBench, a 2.3K+ single-choice benchmark for CSC reasoning, instantiated through our PVC 3D Evaluation Framework (Process, Variety, and Cognition). The Process axis aligns tasks with SCOR+Enable; the Variety axis operationalizes commodity-specific rule systems under coupled material-information-financial constraints, grounded in authoritative exchange guidebooks/rulebooks and industry reports; and the Cognition axis follows Bloom's revised taxonomy. Evaluating representative LLMs under a direct prompting setting, we observe strong performance on the Process and Cognition axes but substantial degradation on the Variety axis, especially on Freight Agreements. CSCBench provides a diagnostic yardstick for measuring and improving LLM capabilities in this high-stakes domain. |
| title | CSCBench: A PVC Diagnostic Benchmark for Commodity Supply Chain Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.01825 |