Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.02172 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910902484205568 |
|---|---|
| author | Huang, Shu-Wei Wu, Xingfang Li, Heng |
| author_facet | Huang, Shu-Wei Wu, Xingfang Li, Heng |
| contents | Large-scale software systems generate vast volumes of system logs that are essential for monitoring, diagnosing, and performance optimization. However, the unstructured nature and ever-growing scale of these logs present significant challenges for manual analysis and automated downstream tasks such as anomaly detection. Log parsing addresses these challenges by converting raw logs into structured formats, enabling efficient log analysis. Despite its importance, existing log parsing methods suffer from limitations in efficiency and scalability, due to the large size of log data and their heterogeneous formats. To overcome these challenges, this study proposes a log parsing approach, LogLSHD, which leverages Locality-Sensitive Hashing (LSH) to group similar logs and integrates Dynamic Time Warping (DTW) to enhance the accuracy of template extraction. LogLSHD demonstrates exceptional efficiency in parsing time, significantly outperforming state-of-the-art methods. For example, compared to Drain, LogLSHD reduces the average parsing time by 73% while increasing the average parsing accuracy by 15% on the LogHub 2.0 benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02172 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LogLSHD: Fast Log Parsing with Locality-Sensitive Hashing and Dynamic Time Warping Huang, Shu-Wei Wu, Xingfang Li, Heng Software Engineering Large-scale software systems generate vast volumes of system logs that are essential for monitoring, diagnosing, and performance optimization. However, the unstructured nature and ever-growing scale of these logs present significant challenges for manual analysis and automated downstream tasks such as anomaly detection. Log parsing addresses these challenges by converting raw logs into structured formats, enabling efficient log analysis. Despite its importance, existing log parsing methods suffer from limitations in efficiency and scalability, due to the large size of log data and their heterogeneous formats. To overcome these challenges, this study proposes a log parsing approach, LogLSHD, which leverages Locality-Sensitive Hashing (LSH) to group similar logs and integrates Dynamic Time Warping (DTW) to enhance the accuracy of template extraction. LogLSHD demonstrates exceptional efficiency in parsing time, significantly outperforming state-of-the-art methods. For example, compared to Drain, LogLSHD reduces the average parsing time by 73% while increasing the average parsing accuracy by 15% on the LogHub 2.0 benchmark. |
| title | LogLSHD: Fast Log Parsing with Locality-Sensitive Hashing and Dynamic Time Warping |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2504.02172 |