DeLog: An Efficient Log Compression Framework with Pattern Signature Synthesis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912839870971904 |
|---|---|
| author | Yu, Siyu Wu, Yifan Xu, Junjielong Fu, Ying Wang, Ning Liu, Maoyin Jiang, Pancheng Zhang, Xiang Jia, Tong He, Pinjia Li, Ying |
| author_facet | Yu, Siyu Wu, Yifan Xu, Junjielong Fu, Ying Wang, Ning Liu, Maoyin Jiang, Pancheng Zhang, Xiang Jia, Tong He, Pinjia Li, Ying |
| contents | Parser-based log compression, which separates static templates from dynamic variables, is a promising approach to exploit the unique structure of log data. However, its performance on complex production logs is often unsatisfactory. This performance gap coincides with a known degradation in the accuracy of its core log parsing component on such data, motivating our investigation into a foundational yet unverified question: does higher parsing accuracy necessarily lead to better compression ratio?
To answer this, we conduct the first empirical study quantifying this relationship and find that a higher parsing accuracy does not guarantee a better compression ratio. Instead, our findings reveal that compression ratio is dictated by achieving effective pattern-based grouping and encoding, i.e., the partitioning of tokens into low entropy, highly compressible groups.
Guided by this insight, we design DeLog, a novel log compressor that implements a Pattern Signature Synthesis mechanism to achieve efficient pattern-based grouping. On 16 public and 10 production datasets, DeLog achieves state-of-the-art compression ratio and speed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_15084 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DeLog: An Efficient Log Compression Framework with Pattern Signature Synthesis Yu, Siyu Wu, Yifan Xu, Junjielong Fu, Ying Wang, Ning Liu, Maoyin Jiang, Pancheng Zhang, Xiang Jia, Tong He, Pinjia Li, Ying Software Engineering Operating Systems Parser-based log compression, which separates static templates from dynamic variables, is a promising approach to exploit the unique structure of log data. However, its performance on complex production logs is often unsatisfactory. This performance gap coincides with a known degradation in the accuracy of its core log parsing component on such data, motivating our investigation into a foundational yet unverified question: does higher parsing accuracy necessarily lead to better compression ratio? To answer this, we conduct the first empirical study quantifying this relationship and find that a higher parsing accuracy does not guarantee a better compression ratio. Instead, our findings reveal that compression ratio is dictated by achieving effective pattern-based grouping and encoding, i.e., the partitioning of tokens into low entropy, highly compressible groups. Guided by this insight, we design DeLog, a novel log compressor that implements a Pattern Signature Synthesis mechanism to achieve efficient pattern-based grouping. On 16 public and 10 production datasets, DeLog achieves state-of-the-art compression ratio and speed. |
| title | DeLog: An Efficient Log Compression Framework with Pattern Signature Synthesis |
| topic | Software Engineering Operating Systems |
| url | https://arxiv.org/abs/2601.15084 |