MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910928647225344 |
|---|---|
| author | Shoshan, Yoel Raboh, Moshiko Ozery-Flato, Michal Ratner, Vadim Golts, Alex Weber, Jeffrey K. Barkan, Ella Rabinovici-Cohen, Simona Polaczek, Sagi Amos, Ido Shapira, Ben Hazan, Liam Ninio, Matan Ravid, Sivan Danziger, Michael M. Shamay, Yosi Kurant, Sharon Morrone, Joseph A. Suryanarayanan, Parthasarathy Rosen-Zvi, Michal Hexter, Efrat |
| author_facet | Shoshan, Yoel Raboh, Moshiko Ozery-Flato, Michal Ratner, Vadim Golts, Alex Weber, Jeffrey K. Barkan, Ella Rabinovici-Cohen, Simona Polaczek, Sagi Amos, Ido Shapira, Ben Hazan, Liam Ninio, Matan Ravid, Sivan Danziger, Michael M. Shamay, Yosi Kurant, Sharon Morrone, Joseph A. Suryanarayanan, Parthasarathy Rosen-Zvi, Michal Hexter, Efrat |
| contents | Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, current models are often siloed, trained separately on small-molecules, proteins, or transcriptomic data, limiting their ability to capture complex, multi-modal interactions. Effective drug discovery requires computational tools that integrate multiple biological entities while supporting prediction and generation, a challenge existing models struggle to address. For this purpose, we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a versatile method applied to create a multi-task foundation model that learns from large-scale biological datasets across diverse modalities, including proteins, small-molecules, and omics. MAMMAL's structured prompt syntax supports classification, regression, and generation tasks while handling token and scalar inputs and outputs. Evaluated on eleven diverse downstream tasks, it reaches a new state of the art (SOTA) in nine tasks and is comparable to SOTA in two tasks, all within a unified architecture, unlike prior task-specific models. Additionally, we explored Alphafold 3 binding prediction capabilities on antibody-antigen and nanobody-antigen complexes showing significantly better classification performance of MAMMAL in 3 out of 4 targets. The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22367 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language Shoshan, Yoel Raboh, Moshiko Ozery-Flato, Michal Ratner, Vadim Golts, Alex Weber, Jeffrey K. Barkan, Ella Rabinovici-Cohen, Simona Polaczek, Sagi Amos, Ido Shapira, Ben Hazan, Liam Ninio, Matan Ravid, Sivan Danziger, Michael M. Shamay, Yosi Kurant, Sharon Morrone, Joseph A. Suryanarayanan, Parthasarathy Rosen-Zvi, Michal Hexter, Efrat Quantitative Methods Artificial Intelligence Machine Learning Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, current models are often siloed, trained separately on small-molecules, proteins, or transcriptomic data, limiting their ability to capture complex, multi-modal interactions. Effective drug discovery requires computational tools that integrate multiple biological entities while supporting prediction and generation, a challenge existing models struggle to address. For this purpose, we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a versatile method applied to create a multi-task foundation model that learns from large-scale biological datasets across diverse modalities, including proteins, small-molecules, and omics. MAMMAL's structured prompt syntax supports classification, regression, and generation tasks while handling token and scalar inputs and outputs. Evaluated on eleven diverse downstream tasks, it reaches a new state of the art (SOTA) in nine tasks and is comparable to SOTA in two tasks, all within a unified architecture, unlike prior task-specific models. Additionally, we explored Alphafold 3 binding prediction capabilities on antibody-antigen and nanobody-antigen complexes showing significantly better classification performance of MAMMAL in 3 out of 4 targets. The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m |
| title | MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language |
| topic | Quantitative Methods Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2410.22367 |