Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.03148 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913975706320896 |
|---|---|
| author | Feng, Yicheng Tan, Xin Sew, Kin Hang Jiang, Yimin Zhu, Yibo Xu, Hong |
| author_facet | Feng, Yicheng Tan, Xin Sew, Kin Hang Jiang, Yimin Zhu, Yibo Xu, Hong |
| contents | Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like prefill/decode (PD) or attention/FFN (AF) for heterogeneous scaling. Existing simulators, architected for co-located, dense models, are unable to capture the intricate system dynamics of these emerging paradigms. We present Frontier, a high-fidelity simulator designed from the ground up for this new landscape. Frontier introduces a unified framework to model both co-located and disaggregated systems, providing native support for MoE inference with expert parallelism (EP). It enables the simulation of complex workflows like cross-cluster expert routing and advanced pipelining strategies for latency hiding. To ensure fidelity and usability, Frontier incorporates refined operator models for improved accuracy. Frontier empowers the community to design and optimize the future of LLM inference at scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03148 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Frontier: Simulating the Next Generation of LLM Inference Systems Feng, Yicheng Tan, Xin Sew, Kin Hang Jiang, Yimin Zhu, Yibo Xu, Hong Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like prefill/decode (PD) or attention/FFN (AF) for heterogeneous scaling. Existing simulators, architected for co-located, dense models, are unable to capture the intricate system dynamics of these emerging paradigms. We present Frontier, a high-fidelity simulator designed from the ground up for this new landscape. Frontier introduces a unified framework to model both co-located and disaggregated systems, providing native support for MoE inference with expert parallelism (EP). It enables the simulation of complex workflows like cross-cluster expert routing and advanced pipelining strategies for latency hiding. To ensure fidelity and usability, Frontier incorporates refined operator models for improved accuracy. Frontier empowers the community to design and optimize the future of LLM inference at scale. |
| title | Frontier: Simulating the Next Generation of LLM Inference Systems |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2508.03148 |