Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.04605 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910687742132224 |
|---|---|
| author | Huang, Haiyu Chen, Cheng Chen, Kunyi Chen, Pengfei Yu, Guangba He, Zilong Wang, Yilun Zhang, Huxing Zhou, Qi |
| author_facet | Huang, Haiyu Chen, Cheng Chen, Kunyi Chen, Pengfei Yu, Guangba He, Zilong Wang, Yilun Zhang, Huxing Zhou, Qi |
| contents | Distributed traces contain valuable information but are often massive in volume, posing a core challenge in tracing framework design: balancing the tradeoff between preserving essential trace information and reducing trace volume. To address this tradeoff, previous approaches typically used a '1 or 0' sampling strategy: retaining sampled traces while completely discarding unsampled ones. However, based on an empirical study on real-world production traces, we discover that the '1 or 0' strategy actually fails to effectively balance this tradeoff.
To achieve a more balanced outcome, we shift the strategy from the '1 or 0' paradigm to the 'commonality + variability' paradigm. The core of 'commonality + variability' paradigm is to first parse traces into common patterns and variable parameters, then aggregate the patterns and filter the parameters. We propose a cost-efficient tracing framework, Mint, which implements the 'commonality + variability' paradigm on the agent side to enable all requests capturing. Our experiments show that Mint can capture all traces and retain more trace information while optimizing trace storage (reduced to an average of 2.7%) and network overhead (reduced to an average of 4.2%). Moreover, experiments also demonstrate that Mint is lightweight enough for production use. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04605 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mint: Cost-Efficient Tracing with All Requests Collection via Commonality and Variability Analysis Huang, Haiyu Chen, Cheng Chen, Kunyi Chen, Pengfei Yu, Guangba He, Zilong Wang, Yilun Zhang, Huxing Zhou, Qi Software Engineering Distributed traces contain valuable information but are often massive in volume, posing a core challenge in tracing framework design: balancing the tradeoff between preserving essential trace information and reducing trace volume. To address this tradeoff, previous approaches typically used a '1 or 0' sampling strategy: retaining sampled traces while completely discarding unsampled ones. However, based on an empirical study on real-world production traces, we discover that the '1 or 0' strategy actually fails to effectively balance this tradeoff. To achieve a more balanced outcome, we shift the strategy from the '1 or 0' paradigm to the 'commonality + variability' paradigm. The core of 'commonality + variability' paradigm is to first parse traces into common patterns and variable parameters, then aggregate the patterns and filter the parameters. We propose a cost-efficient tracing framework, Mint, which implements the 'commonality + variability' paradigm on the agent side to enable all requests capturing. Our experiments show that Mint can capture all traces and retain more trace information while optimizing trace storage (reduced to an average of 2.7%) and network overhead (reduced to an average of 4.2%). Moreover, experiments also demonstrate that Mint is lightweight enough for production use. |
| title | Mint: Cost-Efficient Tracing with All Requests Collection via Commonality and Variability Analysis |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2411.04605 |