CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909022688378880 |
|---|---|
| author | Ding, Eric Oh, Byungsoo Kataria, Bhaskar Guo, Kaiwen Gvero, Jelena Kumar, Abhishek Vijaya Devraj, Arjun Bowen, Lindsey Sonwane, Atharv Manzoor, Emaad Singh, Rachee |
| author_facet | Ding, Eric Oh, Byungsoo Kataria, Bhaskar Guo, Kaiwen Gvero, Jelena Kumar, Abhishek Vijaya Devraj, Arjun Bowen, Lindsey Sonwane, Atharv Manzoor, Emaad Singh, Rachee |
| contents | Evaluative claims about LLM infrastructure -- ``workload X is fastest on hardware Y with software Z'' -- depend on a complex configuration space spanning hardware accelerators, interconnect bandwidth, software frameworks, parallelism plans, and communication libraries. Current infrastructure evaluation benchmarks publish a small set of end-to-end numbers that do not explain why one configuration outperforms another. We present CCL-Bench, a trace-based benchmark that addresses the limitations of existing benchmarks by recording reusable evidence for every ML workload. Each contributed data point in CCL-Bench packages an execution trace, a YAML workload card, and the launch scripts. We have developed a community-extensible toolkit to compute fine-grained compute, memory, and communication efficiency metrics from this evidence. Using CCL-Bench, we surface three claims that summary-statistic benchmarks cannot support: (i) higher compute-communication overlap can coincide with longer training step time and reveal inefficient parallelization choices, (ii) doubling TPU interconnect bandwidth yields a much higher end-to-end improvement in step time than doubling GPU interconnect bandwidth on small and medium workloads, and (iii) the best-tuned configuration on one training framework can run up to 3$\times$ slower than the best-tuned configuration on a peer framework on identical hardware. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06544 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure Ding, Eric Oh, Byungsoo Kataria, Bhaskar Guo, Kaiwen Gvero, Jelena Kumar, Abhishek Vijaya Devraj, Arjun Bowen, Lindsey Sonwane, Atharv Manzoor, Emaad Singh, Rachee Distributed, Parallel, and Cluster Computing Networking and Internet Architecture Evaluative claims about LLM infrastructure -- ``workload X is fastest on hardware Y with software Z'' -- depend on a complex configuration space spanning hardware accelerators, interconnect bandwidth, software frameworks, parallelism plans, and communication libraries. Current infrastructure evaluation benchmarks publish a small set of end-to-end numbers that do not explain why one configuration outperforms another. We present CCL-Bench, a trace-based benchmark that addresses the limitations of existing benchmarks by recording reusable evidence for every ML workload. Each contributed data point in CCL-Bench packages an execution trace, a YAML workload card, and the launch scripts. We have developed a community-extensible toolkit to compute fine-grained compute, memory, and communication efficiency metrics from this evidence. Using CCL-Bench, we surface three claims that summary-statistic benchmarks cannot support: (i) higher compute-communication overlap can coincide with longer training step time and reveal inefficient parallelization choices, (ii) doubling TPU interconnect bandwidth yields a much higher end-to-end improvement in step time than doubling GPU interconnect bandwidth on small and medium workloads, and (iii) the best-tuned configuration on one training framework can run up to 3$\times$ slower than the best-tuned configuration on a peer framework on identical hardware. |
| title | CCL-Bench 1.0: A Trace-Based Benchmark for LLM Infrastructure |
| topic | Distributed, Parallel, and Cluster Computing Networking and Internet Architecture |
| url | https://arxiv.org/abs/2605.06544 |