Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908274595463168 |
|---|---|
| author | Zarkadas, Ioannis Tomlinson, Amanda Cidon, Asaf Kasikci, Baris Weisse, Ofir |
| author_facet | Zarkadas, Ioannis Tomlinson, Amanda Cidon, Asaf Kasikci, Baris Weisse, Ofir |
| contents | As models become larger, ML accelerators are a scarce resource whose performance must be continually optimized to improve efficiency. Existing performance analysis tools are coarse grained, and fail to capture model performance at the machine-code level. In addition, these tools often do not provide specific recommendations for optimizations. We present xPU-Shark, a fine-grained methodology for analyzing ML models at the machine-code level that provides actionable optimization suggestions. Our core insight is to use a hardware-level simulator, an artifact of the hardware design process that we can re-purpose for performance analysis. xPU-Shark captures traces from production deployments running on accelerators and replays them in a modified microarchitecture simulator to gain low-level insights into the model's performance. We implement xPU-Shark for our in-house accelerator and used it to analyze the performance of several of our production LLMs, revealing several previously-unknown microarchitecture inefficiencies. Leveraging these insights, we optimize a common communication collective by up to 15% and reduce token generation latency by up to 4.1%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14781 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation Zarkadas, Ioannis Tomlinson, Amanda Cidon, Asaf Kasikci, Baris Weisse, Ofir Performance Distributed, Parallel, and Cluster Computing Machine Learning As models become larger, ML accelerators are a scarce resource whose performance must be continually optimized to improve efficiency. Existing performance analysis tools are coarse grained, and fail to capture model performance at the machine-code level. In addition, these tools often do not provide specific recommendations for optimizations. We present xPU-Shark, a fine-grained methodology for analyzing ML models at the machine-code level that provides actionable optimization suggestions. Our core insight is to use a hardware-level simulator, an artifact of the hardware design process that we can re-purpose for performance analysis. xPU-Shark captures traces from production deployments running on accelerators and replays them in a modified microarchitecture simulator to gain low-level insights into the model's performance. We implement xPU-Shark for our in-house accelerator and used it to analyze the performance of several of our production LLMs, revealing several previously-unknown microarchitecture inefficiencies. Leveraging these insights, we optimize a common communication collective by up to 15% and reduce token generation latency by up to 4.1%. |
| title | Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation |
| topic | Performance Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2503.14781 |