Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.12409 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912646090981376 |
|---|---|
| author | Liu, Yunuo Zhu, Dawei Al-Khalili, Zena Cheng, Dai Chen, Yanjun Klakow, Dietrich Zhang, Wei Shen, Xiaoyu |
| author_facet | Liu, Yunuo Zhu, Dawei Al-Khalili, Zena Cheng, Dai Chen, Yanjun Klakow, Dietrich Zhang, Wei Shen, Xiaoyu |
| contents | We present PricingLogic, the first benchmark that probes whether Large Language Models(LLMs) can reliably automate tourism-related prices when multiple, overlapping fare rules apply. Travel agencies are eager to offload this error-prone task onto AI systems; however, deploying LLMs without verified reliability could result in significant financial losses and erode customer trust. PricingLogic comprises 300 natural-language questions based on booking requests derived from 42 real-world pricing policies, spanning two levels of difficulty: (i) basic customer-type pricing and (ii)bundled-tour calculations involving interacting discounts. Evaluations of a line of LLMs reveal a steep performance drop on the harder tier,exposing systematic failures in rule interpretation and arithmetic reasoning.These results highlight that, despite their general capabilities, today's LLMs remain unreliable in revenue-critical applications without further safeguards or domain adaptation. Our code and dataset are available at https://github.com/EIT-NLP/PricingLogic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12409 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks Liu, Yunuo Zhu, Dawei Al-Khalili, Zena Cheng, Dai Chen, Yanjun Klakow, Dietrich Zhang, Wei Shen, Xiaoyu Artificial Intelligence We present PricingLogic, the first benchmark that probes whether Large Language Models(LLMs) can reliably automate tourism-related prices when multiple, overlapping fare rules apply. Travel agencies are eager to offload this error-prone task onto AI systems; however, deploying LLMs without verified reliability could result in significant financial losses and erode customer trust. PricingLogic comprises 300 natural-language questions based on booking requests derived from 42 real-world pricing policies, spanning two levels of difficulty: (i) basic customer-type pricing and (ii)bundled-tour calculations involving interacting discounts. Evaluations of a line of LLMs reveal a steep performance drop on the harder tier,exposing systematic failures in rule interpretation and arithmetic reasoning.These results highlight that, despite their general capabilities, today's LLMs remain unreliable in revenue-critical applications without further safeguards or domain adaptation. Our code and dataset are available at https://github.com/EIT-NLP/PricingLogic. |
| title | PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.12409 |