MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910300296445952 |
|---|---|
| author | Liu, Zhiyuan Li, Sihang Luo, Yanchen Fei, Hao Cao, Yixin Kawaguchi, Kenji Wang, Xiang Chua, Tat-Seng |
| author_facet | Liu, Zhiyuan Li, Sihang Luo, Yanchen Fei, Hao Cao, Yixin Kawaguchi, Kenji Wang, Xiang Chua, Tat-Seng |
| contents | Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12798 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter Liu, Zhiyuan Li, Sihang Luo, Yanchen Fei, Hao Cao, Yixin Kawaguchi, Kenji Wang, Xiang Chua, Tat-Seng Computation and Language Multimedia Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA. |
| title | MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter |
| topic | Computation and Language Multimedia |
| url | https://arxiv.org/abs/2310.12798 |