Does Using Counterfactual Help LLMs Explain Textual Importance in Classification?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915533696270336 |
|---|---|
| author | Tan, Nelvin Cheung, James Asikin Shih, Yu-Ching Yang, Dong Salunkhe, Amol |
| author_facet | Tan, Nelvin Cheung, James Asikin Shih, Yu-Ching Yang, Dong Salunkhe, Amol |
| contents | Large language models (LLMs) are becoming useful in many domains due to their impressive abilities that arise from large training datasets and large model sizes. More recently, they have been shown to be very effective in textual classification tasks, motivating the need to explain the LLMs' decisions. Motivated by practical constrains where LLMs are black-boxed and LLM calls are expensive, we study how incorporating counterfactuals into LLM reasoning can affect the LLM's ability to identify the top words that have contributed to its classification decision. To this end, we introduce a framework called the decision changing rate that helps us quantify the importance of the top words in classification. Our experimental results show that using counterfactuals can be helpful. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04031 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Does Using Counterfactual Help LLMs Explain Textual Importance in Classification? Tan, Nelvin Cheung, James Asikin Shih, Yu-Ching Yang, Dong Salunkhe, Amol Computation and Language Artificial Intelligence Large language models (LLMs) are becoming useful in many domains due to their impressive abilities that arise from large training datasets and large model sizes. More recently, they have been shown to be very effective in textual classification tasks, motivating the need to explain the LLMs' decisions. Motivated by practical constrains where LLMs are black-boxed and LLM calls are expensive, we study how incorporating counterfactuals into LLM reasoning can affect the LLM's ability to identify the top words that have contributed to its classification decision. To this end, we introduce a framework called the decision changing rate that helps us quantify the importance of the top words in classification. Our experimental results show that using counterfactuals can be helpful. |
| title | Does Using Counterfactual Help LLMs Explain Textual Importance in Classification? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.04031 |