AD-LLM: Benchmarking Large Language Models for Anomaly Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911200470630400 |
|---|---|
| author | Yang, Tiankai Nian, Yi Li, Shawn Xu, Ruiyao Li, Yuangang Li, Jiaqi Xiao, Zhuo Hu, Xiyang Rossi, Ryan Ding, Kaize Hu, Xia Zhao, Yue |
| author_facet | Yang, Tiankai Nian, Yi Li, Shawn Xu, Ruiyao Li, Yuangang Li, Jiaqi Xiao, Zhuo Hu, Xiyang Rossi, Ryan Ding, Kaize Hu, Xia Zhao, Yue |
| contents | Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. Within natural language processing (NLP), AD helps detect issues like spam, misinformation, and unusual user activity. Although large language models (LLMs) have had a strong impact on tasks such as text generation and summarization, their potential in AD has not been studied enough. This paper introduces AD-LLM, the first benchmark that evaluates how LLMs can help with NLP anomaly detection. We examine three key tasks: (i) zero-shot detection, using LLMs' pre-trained knowledge to perform AD without tasks-specific training; (ii) data augmentation, generating synthetic data and category descriptions to improve AD models; and (iii) model selection, using LLMs to suggest unsupervised AD models. Through experiments with different datasets, we find that LLMs can work well in zero-shot AD, that carefully designed augmentation methods are useful, and that explaining model selection for specific datasets remains challenging. Based on these results, we outline six future research directions on LLMs for AD. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11142 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AD-LLM: Benchmarking Large Language Models for Anomaly Detection Yang, Tiankai Nian, Yi Li, Shawn Xu, Ruiyao Li, Yuangang Li, Jiaqi Xiao, Zhuo Hu, Xiyang Rossi, Ryan Ding, Kaize Hu, Xia Zhao, Yue Computation and Language Artificial Intelligence Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. Within natural language processing (NLP), AD helps detect issues like spam, misinformation, and unusual user activity. Although large language models (LLMs) have had a strong impact on tasks such as text generation and summarization, their potential in AD has not been studied enough. This paper introduces AD-LLM, the first benchmark that evaluates how LLMs can help with NLP anomaly detection. We examine three key tasks: (i) zero-shot detection, using LLMs' pre-trained knowledge to perform AD without tasks-specific training; (ii) data augmentation, generating synthetic data and category descriptions to improve AD models; and (iii) model selection, using LLMs to suggest unsupervised AD models. Through experiments with different datasets, we find that LLMs can work well in zero-shot AD, that carefully designed augmentation methods are useful, and that explaining model selection for specific datasets remains challenging. Based on these results, we outline six future research directions on LLMs for AD. |
| title | AD-LLM: Benchmarking Large Language Models for Anomaly Detection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.11142 |