Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.23836 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911182164590592 |
|---|---|
| author | Zhou, Chenyu Shi, Xiaoming Qiu, Hui Zheng, Xiawu Leng, Haitao Jiang, Yankai Liu, Shaoguo Gao, Tingting Ji, Rongrong |
| author_facet | Zhou, Chenyu Shi, Xiaoming Qiu, Hui Zheng, Xiawu Leng, Haitao Jiang, Yankai Liu, Shaoguo Gao, Tingting Ji, Rongrong |
| contents | E-commerce agents contribute greatly to helping users complete their e-commerce needs. To promote further research and application of e-commerce agents, benchmarking frameworks are introduced for evaluating LLM agents in the e-commerce domain. Despite the progress, current benchmarks lack evaluating agents' capability to handle mixed-type e-commerce dialogue and complex domain rules. To address the issue, this work first introduces a novel corpus, termed Mix-ECom, which is constructed based on real-world customer-service dialogues with post-processing to remove user privacy and add CoT process. Specifically, Mix-ECom contains 4,799 samples with multiply dialogue types in each e-commerce dialogue, covering four dialogue types (QA, recommendation, task-oriented dialogue, and chit-chat), three e-commerce task types (pre-sales, logistics, after-sales), and 82 e-commerce rules. Furthermore, this work build baselines on Mix-Ecom and propose a dynamic framework to further improve the performance. Results show that current e-commerce agents lack sufficient capabilities to handle e-commerce dialogues, due to the hallucination cased by complex domain rules. The dataset will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23836 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mix-Ecom: Towards Mixed-Type E-Commerce Dialogues with Complex Domain Rules Zhou, Chenyu Shi, Xiaoming Qiu, Hui Zheng, Xiawu Leng, Haitao Jiang, Yankai Liu, Shaoguo Gao, Tingting Ji, Rongrong Artificial Intelligence E-commerce agents contribute greatly to helping users complete their e-commerce needs. To promote further research and application of e-commerce agents, benchmarking frameworks are introduced for evaluating LLM agents in the e-commerce domain. Despite the progress, current benchmarks lack evaluating agents' capability to handle mixed-type e-commerce dialogue and complex domain rules. To address the issue, this work first introduces a novel corpus, termed Mix-ECom, which is constructed based on real-world customer-service dialogues with post-processing to remove user privacy and add CoT process. Specifically, Mix-ECom contains 4,799 samples with multiply dialogue types in each e-commerce dialogue, covering four dialogue types (QA, recommendation, task-oriented dialogue, and chit-chat), three e-commerce task types (pre-sales, logistics, after-sales), and 82 e-commerce rules. Furthermore, this work build baselines on Mix-Ecom and propose a dynamic framework to further improve the performance. Results show that current e-commerce agents lack sufficient capabilities to handle e-commerce dialogues, due to the hallucination cased by complex domain rules. The dataset will be publicly available. |
| title | Mix-Ecom: Towards Mixed-Type E-Commerce Dialogues with Complex Domain Rules |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.23836 |