Concept Gradient: Concept-based Interpretation Without Linear Assumption
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917582666203136 |
|---|---|
| author | Bai, Andrew Yeh, Chih-Kuan Ravikumar, Pradeep Lin, Neil Y. C. Hsieh, Cho-Jui |
| author_facet | Bai, Andrew Yeh, Chih-Kuan Ravikumar, Pradeep Lin, Neil Y. C. Hsieh, Cho-Jui |
| contents | Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based interpretation is Concept Activation Vector (CAV). CAV relies on learning a linear relation between some latent representation of a given model and concepts. The linear separability is usually implicitly assumed but does not hold true in general. In this work, we started from the original intent of concept-based interpretation and proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space. We demonstrated empirically that CG outperforms CAV in both toy examples and real world datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_14966 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Concept Gradient: Concept-based Interpretation Without Linear Assumption Bai, Andrew Yeh, Chih-Kuan Ravikumar, Pradeep Lin, Neil Y. C. Hsieh, Cho-Jui Machine Learning Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based interpretation is Concept Activation Vector (CAV). CAV relies on learning a linear relation between some latent representation of a given model and concepts. The linear separability is usually implicitly assumed but does not hold true in general. In this work, we started from the original intent of concept-based interpretation and proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space. We demonstrated empirically that CG outperforms CAV in both toy examples and real world datasets. |
| title | Concept Gradient: Concept-based Interpretation Without Linear Assumption |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2208.14966 |