Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918129634902016 |
|---|---|
| author | Zhao, Yunxiao Xu, Hao Wang, Zhiqiang Li, Xiaoli Liang, Jiye Li, Ru |
| author_facet | Zhao, Yunxiao Xu, Hao Wang, Zhiqiang Li, Xiaoli Liang, Jiye Li, Ru |
| contents | Pre-trained Language Models (PLMs) are trained on large amounts of unlabeled data, yet they exhibit remarkable reasoning skills. However, the trustworthiness challenges posed by these black-box models have become increasingly evident in recent years. To alleviate this problem, this paper proposes a novel Knowledge-guided Probing approach called KnowProb in a post-hoc explanation way, which aims to probe whether black-box PLMs understand implicit knowledge beyond the given text, rather than focusing only on the surface level content of the text. We provide six potential explanations derived from the underlying content of the given text, including three knowledge-based understanding and three association-based reasoning. In experiments, we validate that current small-scale (or large-scale) PLMs only learn a single distribution of representation, and still face significant challenges in capturing the hidden knowledge behind a given text. Furthermore, we demonstrate that our proposed approach is effective for identifying the limitations of existing black-box models from multiple probing perspectives, which facilitates researchers to promote the study of detecting black-box models in an explainable way. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16969 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective Zhao, Yunxiao Xu, Hao Wang, Zhiqiang Li, Xiaoli Liang, Jiye Li, Ru Computation and Language Artificial Intelligence Databases Pre-trained Language Models (PLMs) are trained on large amounts of unlabeled data, yet they exhibit remarkable reasoning skills. However, the trustworthiness challenges posed by these black-box models have become increasingly evident in recent years. To alleviate this problem, this paper proposes a novel Knowledge-guided Probing approach called KnowProb in a post-hoc explanation way, which aims to probe whether black-box PLMs understand implicit knowledge beyond the given text, rather than focusing only on the surface level content of the text. We provide six potential explanations derived from the underlying content of the given text, including three knowledge-based understanding and three association-based reasoning. In experiments, we validate that current small-scale (or large-scale) PLMs only learn a single distribution of representation, and still face significant challenges in capturing the hidden knowledge behind a given text. Furthermore, we demonstrate that our proposed approach is effective for identifying the limitations of existing black-box models from multiple probing perspectives, which facilitates researchers to promote the study of detecting black-box models in an explainable way. |
| title | Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective |
| topic | Computation and Language Artificial Intelligence Databases |
| url | https://arxiv.org/abs/2508.16969 |