Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914747766538240 |
|---|---|
| author | Zhang, Xiaokang Yao, Zijun Zhang, Jing Yun, Kaifeng Yu, Jifan Li, Juanzi Tang, Jie |
| author_facet | Zhang, Xiaokang Yao, Zijun Zhang, Jing Yun, Kaifeng Yu, Jifan Li, Juanzi Tang, Jie |
| contents | Detecting non-factual content is a longstanding goal to increase the trustworthiness of large language models (LLMs) generations. Current factuality probes, trained using humanannotated labels, exhibit limited transferability to out-of-distribution content, while online selfconsistency checking imposes extensive computation burden due to the necessity of generating multiple outputs. This paper proposes PINOSE, which trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data and achieving transferability across diverse data distributions. As the consistency check process is offline, PINOSE reduces the computational burden of generating multiple responses by online consistency verification. Additionally, it examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies. Experiment results on both factuality detection and question answering benchmarks show that PINOSE achieves surpassing results than existing factuality detection methods. Our code and datasets are publicly available on this anonymized repository. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06742 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking Zhang, Xiaokang Yao, Zijun Zhang, Jing Yun, Kaifeng Yu, Jifan Li, Juanzi Tang, Jie Computation and Language Detecting non-factual content is a longstanding goal to increase the trustworthiness of large language models (LLMs) generations. Current factuality probes, trained using humanannotated labels, exhibit limited transferability to out-of-distribution content, while online selfconsistency checking imposes extensive computation burden due to the necessity of generating multiple outputs. This paper proposes PINOSE, which trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data and achieving transferability across diverse data distributions. As the consistency check process is offline, PINOSE reduces the computational burden of generating multiple responses by online consistency verification. Additionally, it examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies. Experiment results on both factuality detection and question answering benchmarks show that PINOSE achieves surpassing results than existing factuality detection methods. Our code and datasets are publicly available on this anonymized repository. |
| title | Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.06742 |