Mapping the Mind of an Instruction-based Image Editing using SMILE
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929643724996608 |
|---|---|
| author | Dehghani, Zeinab Aslansefat, Koorosh Khan, Adil Rivera, Adín Ramírez George, Franky Khalid, Muhammad |
| author_facet | Dehghani, Zeinab Aslansefat, Koorosh Khan, Adil Rivera, Adín Ramírez George, Franky Khalid, Muhammad |
| contents | Despite recent advancements in Instruct-based Image Editing models for generating high-quality images, they are known as black boxes and a significant barrier to transparency and user trust. To solve this issue, we introduce SMILE (Statistical Model-agnostic Interpretability with Local Explanations), a novel model-agnostic for localized interpretability that provides a visual heatmap to clarify the textual elements' influence on image-generating models. We applied our method to various Instruction-based Image Editing models like Pix2Pix, Image2Image-turbo and Diffusers-Inpaint and showed how our model can improve interpretability and reliability. Also, we use stability, accuracy, fidelity, and consistency metrics to evaluate our method. These findings indicate the exciting potential of model-agnostic interpretability for reliability and trustworthiness in critical applications such as healthcare and autonomous driving while encouraging additional investigation into the significance of interpretability in enhancing dependable image editing models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_16277 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mapping the Mind of an Instruction-based Image Editing using SMILE Dehghani, Zeinab Aslansefat, Koorosh Khan, Adil Rivera, Adín Ramírez George, Franky Khalid, Muhammad Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction Despite recent advancements in Instruct-based Image Editing models for generating high-quality images, they are known as black boxes and a significant barrier to transparency and user trust. To solve this issue, we introduce SMILE (Statistical Model-agnostic Interpretability with Local Explanations), a novel model-agnostic for localized interpretability that provides a visual heatmap to clarify the textual elements' influence on image-generating models. We applied our method to various Instruction-based Image Editing models like Pix2Pix, Image2Image-turbo and Diffusers-Inpaint and showed how our model can improve interpretability and reliability. Also, we use stability, accuracy, fidelity, and consistency metrics to evaluate our method. These findings indicate the exciting potential of model-agnostic interpretability for reliability and trustworthiness in critical applications such as healthcare and autonomous driving while encouraging additional investigation into the significance of interpretability in enhancing dependable image editing models. |
| title | Mapping the Mind of an Instruction-based Image Editing using SMILE |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction |
| url | https://arxiv.org/abs/2412.16277 |