Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.17451 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913875466649600 |
|---|---|
| author | Peng, Yifei Zha, Zijie Jin, Yu Luo, Zhexu Dai, Wang-Zhou Ren, Zhong Ding, Yao-Xiang Zhou, Kun |
| author_facet | Peng, Yifei Zha, Zijie Jin, Yu Luo, Zhexu Dai, Wang-Zhou Ren, Zhong Ding, Yao-Xiang Zhou, Kun |
| contents | Making neural visual generative models controllable by logical reasoning systems is promising for improving faithfulness, transparency, and generalizability. We propose the Abductive visual Generation (AbdGen) approach to build such logic-integrated models. A vector-quantized symbol grounding mechanism and the corresponding disentanglement training method are introduced to enhance the controllability of logical symbols over generation. Furthermore, we propose two logical abduction methods to make our approach require few labeled training data and support the induction of latent logical generative rules from data. We experimentally show that our approach can be utilized to integrate various neural generative models with logical reasoning systems, by both learning from scratch or utilizing pre-trained models directly. The code is released at https://github.com/future-item/AbdGen. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_17451 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings Peng, Yifei Zha, Zijie Jin, Yu Luo, Zhexu Dai, Wang-Zhou Ren, Zhong Ding, Yao-Xiang Zhou, Kun Artificial Intelligence Computer Vision and Pattern Recognition Graphics Making neural visual generative models controllable by logical reasoning systems is promising for improving faithfulness, transparency, and generalizability. We propose the Abductive visual Generation (AbdGen) approach to build such logic-integrated models. A vector-quantized symbol grounding mechanism and the corresponding disentanglement training method are introduced to enhance the controllability of logical symbols over generation. Furthermore, we propose two logical abduction methods to make our approach require few labeled training data and support the induction of latent logical generative rules from data. We experimentally show that our approach can be utilized to integrate various neural generative models with logical reasoning systems, by both learning from scratch or utilizing pre-trained models directly. The code is released at https://github.com/future-item/AbdGen. |
| title | Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2310.17451 |