Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.17256 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908602779828224 |
|---|---|
| author | Atakishiyev, Shahin Babiker, Housam K. B. Dai, Jiayi Farruque, Nawshad Hayashi, Teruaki Hriti, Nafisa Sadaf Rahman, Md Abed Smith, Iain Kim, Mi-Young Zaïane, Osmar R. Goebel, Randy |
| author_facet | Atakishiyev, Shahin Babiker, Housam K. B. Dai, Jiayi Farruque, Nawshad Hayashi, Teruaki Hriti, Nafisa Sadaf Rahman, Md Abed Smith, Iain Kim, Mi-Young Zaïane, Osmar R. Goebel, Randy |
| contents | Large language models have exhibited impressive performance across a broad range of downstream tasks in natural language processing. However, how a language model predicts the next token and generates content is not generally understandable by humans. Furthermore, these models often make errors in prediction and reasoning, known as hallucinations. These errors underscore the urgent need to better understand and interpret the intricate inner workings of language models and how they generate predictive outputs. Motivated by this gap, this paper investigates local explainability and mechanistic interpretability within Transformer-based large language models to foster trust in such models. In this regard, our paper aims to make three key contributions. First, we present a review of local explainability and mechanistic interpretability approaches and insights from relevant studies in the literature. Furthermore, we describe experimental studies on explainability and reasoning with large language models in two critical domains -- healthcare and autonomous driving -- and analyze the trust implications of such explanations for explanation receivers. Finally, we summarize current unaddressed issues in the evolving landscape of LLM explainability and outline the opportunities, critical challenges, and future directions toward generating human-aligned, trustworthy LLM explanations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_17256 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations Atakishiyev, Shahin Babiker, Housam K. B. Dai, Jiayi Farruque, Nawshad Hayashi, Teruaki Hriti, Nafisa Sadaf Rahman, Md Abed Smith, Iain Kim, Mi-Young Zaïane, Osmar R. Goebel, Randy Computation and Language Large language models have exhibited impressive performance across a broad range of downstream tasks in natural language processing. However, how a language model predicts the next token and generates content is not generally understandable by humans. Furthermore, these models often make errors in prediction and reasoning, known as hallucinations. These errors underscore the urgent need to better understand and interpret the intricate inner workings of language models and how they generate predictive outputs. Motivated by this gap, this paper investigates local explainability and mechanistic interpretability within Transformer-based large language models to foster trust in such models. In this regard, our paper aims to make three key contributions. First, we present a review of local explainability and mechanistic interpretability approaches and insights from relevant studies in the literature. Furthermore, we describe experimental studies on explainability and reasoning with large language models in two critical domains -- healthcare and autonomous driving -- and analyze the trust implications of such explanations for explanation receivers. Finally, we summarize current unaddressed issues in the evolving landscape of LLM explainability and outline the opportunities, critical challenges, and future directions toward generating human-aligned, trustworthy LLM explanations. |
| title | Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.17256 |