Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916486440812544 |
|---|---|
| author | Morales-Pastor, Adrián Vázquez-Reza, Raquel Wieczór, Miłosz Valverde, Clàudia Gil-Sorribes, Manel Miquel-Oliver, Bertran Ciudad, Álvaro Molina, Alexis |
| author_facet | Morales-Pastor, Adrián Vázquez-Reza, Raquel Wieczór, Miłosz Valverde, Clàudia Gil-Sorribes, Manel Miquel-Oliver, Bertran Ciudad, Álvaro Molina, Alexis |
| contents | RNA is a vital biomolecule with numerous roles and functions within cells, and interest in targeting it for therapeutic purposes has grown significantly in recent years. However, fully understanding and predicting RNA behavior, particularly for applications in drug discovery, remains a challenge due to the complexity of RNA structures and interactions. While foundational models in biology have demonstrated success in modeling several biomolecules, especially proteins, achieving similar breakthroughs for RNA has proven more difficult. Current RNA models have yet to match the performance observed in the protein domain, leaving an important gap in computational biology. In this work, we present ChaRNABERT, a suite of sample and parameter-efficient RNA foundational models, that through a learnable tokenization process, are able to reach state-of-the-art performance on several tasks in established benchmarks. We extend its testing in relevant downstream tasks such as RNA-protein and aptamer-protein interaction prediction. Weights and inference code for ChaRNABERT-8M will be provided for academic research use. The other models will be available upon request. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11808 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models Morales-Pastor, Adrián Vázquez-Reza, Raquel Wieczór, Miłosz Valverde, Clàudia Gil-Sorribes, Manel Miquel-Oliver, Bertran Ciudad, Álvaro Molina, Alexis Quantitative Methods Artificial Intelligence Machine Learning Biomolecules RNA is a vital biomolecule with numerous roles and functions within cells, and interest in targeting it for therapeutic purposes has grown significantly in recent years. However, fully understanding and predicting RNA behavior, particularly for applications in drug discovery, remains a challenge due to the complexity of RNA structures and interactions. While foundational models in biology have demonstrated success in modeling several biomolecules, especially proteins, achieving similar breakthroughs for RNA has proven more difficult. Current RNA models have yet to match the performance observed in the protein domain, leaving an important gap in computational biology. In this work, we present ChaRNABERT, a suite of sample and parameter-efficient RNA foundational models, that through a learnable tokenization process, are able to reach state-of-the-art performance on several tasks in established benchmarks. We extend its testing in relevant downstream tasks such as RNA-protein and aptamer-protein interaction prediction. Weights and inference code for ChaRNABERT-8M will be provided for academic research use. The other models will be available upon request. |
| title | Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models |
| topic | Quantitative Methods Artificial Intelligence Machine Learning Biomolecules |
| url | https://arxiv.org/abs/2411.11808 |