TokenShapley: Token Level Context Attribution with Shapley Value
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912472757174272 |
|---|---|
| author | Xiao, Yingtai Zhu, Yuqing Samyoun, Sirat Zhang, Wanrong Wang, Jiachen T. Du, Jian |
| author_facet | Xiao, Yingtai Zhu, Yuqing Samyoun, Sirat Zhang, Wanrong Wang, Jiachen T. Du, Jian |
| contents | Large language models (LLMs) demonstrate strong capabilities in in-context learning, but verifying the correctness of their generated responses remains a challenge. Prior work has explored attribution at the sentence level, but these methods fall short when users seek attribution for specific keywords within the response, such as numbers, years, or names. To address this limitation, we propose TokenShapley, a novel token-level attribution method that combines Shapley value-based data attribution with KNN-based retrieval techniques inspired by recent advances in KNN-augmented LLMs. By leveraging a precomputed datastore for contextual retrieval and computing Shapley values to quantify token importance, TokenShapley provides a fine-grained data attribution approach. Extensive evaluations on four benchmarks show that TokenShapley outperforms state-of-the-art baselines in token-level attribution, achieving an 11-23% improvement in accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05261 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TokenShapley: Token Level Context Attribution with Shapley Value Xiao, Yingtai Zhu, Yuqing Samyoun, Sirat Zhang, Wanrong Wang, Jiachen T. Du, Jian Computation and Language Machine Learning Large language models (LLMs) demonstrate strong capabilities in in-context learning, but verifying the correctness of their generated responses remains a challenge. Prior work has explored attribution at the sentence level, but these methods fall short when users seek attribution for specific keywords within the response, such as numbers, years, or names. To address this limitation, we propose TokenShapley, a novel token-level attribution method that combines Shapley value-based data attribution with KNN-based retrieval techniques inspired by recent advances in KNN-augmented LLMs. By leveraging a precomputed datastore for contextual retrieval and computing Shapley values to quantify token importance, TokenShapley provides a fine-grained data attribution approach. Extensive evaluations on four benchmarks show that TokenShapley outperforms state-of-the-art baselines in token-level attribution, achieving an 11-23% improvement in accuracy. |
| title | TokenShapley: Token Level Context Attribution with Shapley Value |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2507.05261 |