Cover Image

LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cui, Tianyu, Ma, Shiyu, Chen, Ziang, Xiao, Tong, Tao, Shimin, Liu, Yilun, Zhang, Shenglin, Lin, Duoming, Liu, Changchang, Cai, Yuzhe, Meng, Weibin, Sun, Yongqian, Pei, Dan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LogPurge: Log Data Purification for Anomaly Detection via Rule-Enhanced Filtering
by: Zhang, Shenglin, et al.
Published: (2025)

LogLM: From Task-based to Instruction-based Automated Log Analysis
by: Liu, Yilun, et al.
Published: (2024)

R-Log: Incentivizing Log Analysis Capability in LLMs via Reasoning-based Reinforcement Learning
by: Liu, Yilun, et al.
Published: (2025)

SuiteEval: Simplifying Retrieval Benchmarks
by: Parry, Andrew, et al.
Published: (2026)

RationAnomaly: Log Anomaly Detection with Rationality via Chain-of-Thought and Reinforcement Learning
by: Xu, Song, et al.
Published: (2025)

Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge
by: Ji, Yuhe, et al.
Published: (2024)

OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Models
by: Liu, Yuhe, et al.
Published: (2023)

SimEval-IR: A Unified Toolkit and Benchmark Suite for Evaluating User Simulators and Search Sessions
by: Zerhoudi, Saber
Published: (2026)

System Log Parsing: A Survey
by: Zhang, Tianzhu, et al.
Published: (2022)

Link Prediction for Event Logs in the Process Industry
by: Zhukova, Anastasia, et al.
Published: (2025)

Similarity-Based Supervised User Session Segmentation Method for Behavior Logs
by: Jin, Yongzhi, et al.
Published: (2025)

Prompt Optimization with Logged Bandit Data
by: Kiyohara, Haruka, et al.
Published: (2025)

Correcting the LogQ Correction: Revisiting Sampled Softmax for Large-Scale Retrieval
by: Khrylchenko, Kirill, et al.
Published: (2025)

IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval
by: Song, Tingyu, et al.
Published: (2025)

OrLog: Resolving Complex Queries with LLMs and Probabilistic Reasoning
by: Hoveyda, Mohanna, et al.
Published: (2026)

LADER: Log-Augmented DEnse Retrieval for Biomedical Literature Search
by: Jin, Qiao, et al.
Published: (2023)

From Logs to Language: Learning Optimal Verbalization for LLM-Based Recommendation at Industry Scale
by: Shi, Yucheng, et al.
Published: (2026)

OPAC User Logs: Implications for Bibliographic Instruction.
by: Kern-Simirenko, Cheryl
Published: (1983)

General-Purpose User Modeling with Behavioral Logs: A Snapchat Case Study
by: Fang, Qixiang, et al.
Published: (2023)

Understanding the Effects of the Baidu-ULTR Logging Policy on Two-Tower Models
by: de Haan, Morris, et al.
Published: (2024)

NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation
by: Zhang, Jiaqi, et al.
Published: (2023)

Object-Centric Analysis of XES Event Logs: Integrating OCED Modeling with SPARQL Queries
by: Latif, Saba, et al.
Published: (2025)

Predictive Modeling: BIM Command Recommendation Based on Large-scale Usage Logs
by: Du, Changyu, et al.
Published: (2025)

SGSimEval: A Comprehensive Multifaceted and Similarity-Enhanced Benchmark for Automatic Survey Generation Systems
by: Guo, Beichen, et al.
Published: (2025)

Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark Suite
by: Thellmann, Klaudia, et al.
Published: (2026)

Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies
by: Liu, Yilun, et al.
Published: (2023)

Harnessing PubMed User Query Logs for Post Hoc Explanations of Recommended Similar Articles
by: Shin, Ashley, et al.
Published: (2024)

ViLLA-MMBench: A Unified Benchmark Suite for LLM-Augmented Multimodal Movie Recommendation
by: Nazary, Fatemeh, et al.
Published: (2025)

Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Prediction
by: Kruff, Andreas Konstantin, et al.
Published: (2025)

HintEval: A Comprehensive Framework for Hint Generation and Evaluation for Questions
by: Mozafari, Jamshid, et al.
Published: (2025)

RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation
by: Meng, Zeyuan, et al.
Published: (2025)

UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis
by: Hui, Yulong, et al.
Published: (2024)

RAIRS: Optimizing Redundant Assignment and List Layout for IVF-Based ANN Search
by: Yang, Zehai, et al.
Published: (2026)

A Benchmark Suite of Reddit-Derived Datasets for Mental Health Detection
by: Hasan, Khalid, et al.
Published: (2026)

DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management
by: Yin, Kai, et al.
Published: (2025)

SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
by: Hu, Tiansheng, et al.
Published: (2026)

LEXA: Legal Case Retrieval via Graph Contrastive Learning with Contextualised LLM Embeddings
by: Tang, Yanran, et al.
Published: (2024)

ABCDE: Application-Based Cluster Diff Evals
by: van Staden, Stephan, et al.
Published: (2024)

Future-Conditioned Recommendations with Multi-Objective Controllable Decision Transformer
by: Gao, Chongming, et al.
Published: (2025)

WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild
by: Deng, Yuntian, et al.
Published: (2024)