SLOFetch: Compressed-Hierarchical Instruction Prefetching for Cloud Microservices
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914169728532480 |
|---|---|
| author | Bao, Zerui Zhu, Di Jiang, Liu Sheng, Shiqi Wang, Ziwei Zhang, Haoyun |
| author_facet | Bao, Zerui Zhu, Di Jiang, Liu Sheng, Shiqi Wang, Ziwei Zhang, Haoyun |
| contents | Large-scale networked services rely on deep soft-ware stacks and microservice orchestration, which increase instruction footprints and create frontend stalls that inflate tail latency and energy. We revisit instruction prefetching for these cloud workloads and present a design that aligns with SLO driven and self optimizing systems. Building on the Entangling Instruction Prefetcher (EIP), we introduce a Compressed Entry that captures up to eight destinations around a base using 36 bits by exploiting spatial clustering, and a Hierarchical Metadata Storage scheme that keeps only L1 resident and frequently queried entries on chip while virtualizing bulk metadata into lower levels. We further add a lightweight Online ML Controller that scores prefetch profitability using context features and a bandit adjusted threshold. On data center applications, our approach preserves EIP like speedups with smaller on chip state and improves efficiency for networked services in the ML era. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04774 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SLOFetch: Compressed-Hierarchical Instruction Prefetching for Cloud Microservices Bao, Zerui Zhu, Di Jiang, Liu Sheng, Shiqi Wang, Ziwei Zhang, Haoyun Machine Learning Hardware Architecture Large-scale networked services rely on deep soft-ware stacks and microservice orchestration, which increase instruction footprints and create frontend stalls that inflate tail latency and energy. We revisit instruction prefetching for these cloud workloads and present a design that aligns with SLO driven and self optimizing systems. Building on the Entangling Instruction Prefetcher (EIP), we introduce a Compressed Entry that captures up to eight destinations around a base using 36 bits by exploiting spatial clustering, and a Hierarchical Metadata Storage scheme that keeps only L1 resident and frequently queried entries on chip while virtualizing bulk metadata into lower levels. We further add a lightweight Online ML Controller that scores prefetch profitability using context features and a bandit adjusted threshold. On data center applications, our approach preserves EIP like speedups with smaller on chip state and improves efficiency for networked services in the ML era. |
| title | SLOFetch: Compressed-Hierarchical Instruction Prefetching for Cloud Microservices |
| topic | Machine Learning Hardware Architecture |
| url | https://arxiv.org/abs/2511.04774 |