LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911900511502336 |
|---|---|
| author | Xu, Junjie Wu, Zongyu Lin, Minhua Zhang, Xiang Wang, Suhang |
| author_facet | Xu, Junjie Wu, Zongyu Lin, Minhua Zhang, Xiang Wang, Suhang |
| contents | Recent progress in Graph Neural Networks (GNNs) has greatly enhanced the ability to model complex molecular structures for predicting properties. Nevertheless, molecular data encompasses more than just graph structures, including textual and visual information that GNNs do not handle well. To bridge this gap, we present an innovative framework that utilizes multimodal molecular data to extract insights from Large Language Models (LLMs). We introduce GALLON (Graph Learning from Large Language Model Distillation), a framework that synergizes the capabilities of LLMs and GNNs by distilling multimodal knowledge into a unified Multilayer Perceptron (MLP). This method integrates the rich textual and visual data of molecules with the structural analysis power of GNNs. Extensive experiments reveal that our distilled MLP model notably improves the accuracy and efficiency of molecular property predictions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01032 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning Xu, Junjie Wu, Zongyu Lin, Minhua Zhang, Xiang Wang, Suhang Machine Learning Artificial Intelligence Recent progress in Graph Neural Networks (GNNs) has greatly enhanced the ability to model complex molecular structures for predicting properties. Nevertheless, molecular data encompasses more than just graph structures, including textual and visual information that GNNs do not handle well. To bridge this gap, we present an innovative framework that utilizes multimodal molecular data to extract insights from Large Language Models (LLMs). We introduce GALLON (Graph Learning from Large Language Model Distillation), a framework that synergizes the capabilities of LLMs and GNNs by distilling multimodal knowledge into a unified Multilayer Perceptron (MLP). This method integrates the rich textual and visual data of molecules with the structural analysis power of GNNs. Extensive experiments reveal that our distilled MLP model notably improves the accuracy and efficiency of molecular property predictions. |
| title | LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.01032 |