Explaining latent representations of generative models with large multimodal models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916211221069824 |
|---|---|
| author | Zhu, Mengdan Liu, Zhenke Pan, Bo Angirekula, Abhinav Zhao, Liang |
| author_facet | Zhu, Mengdan Liu, Zhenke Pan, Bo Angirekula, Abhinav Zhao, Liang |
| contents | Learning interpretable representations of data generative latent factors is an important topic for the development of artificial intelligence. With the rise of the large multimodal model, it can align images with text to generate answers. In this work, we propose a framework to comprehensively explain each latent variable in the generative models using a large multimodal model. We further measure the uncertainty of our generated explanations, quantitatively evaluate the performance of explanation generation among multiple large multimodal models, and qualitatively visualize the variations of each latent variable to learn the disentanglement effects of different generative models on explanations. Finally, we discuss the explanatory capabilities and limitations of state-of-the-art large multimodal models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01858 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explaining latent representations of generative models with large multimodal models Zhu, Mengdan Liu, Zhenke Pan, Bo Angirekula, Abhinav Zhao, Liang Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Learning interpretable representations of data generative latent factors is an important topic for the development of artificial intelligence. With the rise of the large multimodal model, it can align images with text to generate answers. In this work, we propose a framework to comprehensively explain each latent variable in the generative models using a large multimodal model. We further measure the uncertainty of our generated explanations, quantitatively evaluate the performance of explanation generation among multiple large multimodal models, and qualitatively visualize the variations of each latent variable to learn the disentanglement effects of different generative models on explanations. Finally, we discuss the explanatory capabilities and limitations of state-of-the-art large multimodal models. |
| title | Explaining latent representations of generative models with large multimodal models |
| topic | Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.01858 |