BEACON: Benchmark for Comprehensive RNA Tasks and Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916519255998464 |
|---|---|
| author | Ren, Yuchen Chen, Zhiyuan Qiao, Lifeng Jing, Hongtai Cai, Yuchen Xu, Sheng Ye, Peng Ma, Xinzhu Sun, Siqi Yan, Hongliang Yuan, Dong Ouyang, Wanli Liu, Xihui |
| author_facet | Ren, Yuchen Chen, Zhiyuan Qiao, Lifeng Jing, Hongtai Cai, Yuchen Xu, Sheng Ye, Peng Ma, Xinzhu Sun, Siqi Yan, Hongliang Yuan, Dong Ouyang, Wanli Liu, Xihui |
| contents | RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms. Despite the emergence of numerous deep learning approaches for RNA, particularly universal RNA language models, there remains a significant lack of standardized benchmarks to assess the effectiveness of these methods. In this study, we introduce the first comprehensive RNA benchmark BEACON (\textbf{BE}nchm\textbf{A}rk for \textbf{CO}mprehensive R\textbf{N}A Task and Language Models). First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications, enabling a comprehensive assessment of the performance of methods on various RNA understanding tasks. Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models. Third, we investigate the vital RNA language model components from the tokenizer and positional encoding aspects. Notably, our findings emphasize the superiority of single nucleotide tokenization and the effectiveness of Attention with Linear Biases (ALiBi) over traditional positional encoding methods. Based on these insights, a simple yet strong baseline called BEACON-B is proposed, which can achieve outstanding performance with limited data and computational resources. The datasets and source code of our benchmark are available at https://github.com/terry-r123/RNABenchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10391 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BEACON: Benchmark for Comprehensive RNA Tasks and Language Models Ren, Yuchen Chen, Zhiyuan Qiao, Lifeng Jing, Hongtai Cai, Yuchen Xu, Sheng Ye, Peng Ma, Xinzhu Sun, Siqi Yan, Hongliang Yuan, Dong Ouyang, Wanli Liu, Xihui Quantitative Methods Machine Learning RNA plays a pivotal role in translating genetic instructions into functional outcomes, underscoring its importance in biological processes and disease mechanisms. Despite the emergence of numerous deep learning approaches for RNA, particularly universal RNA language models, there remains a significant lack of standardized benchmarks to assess the effectiveness of these methods. In this study, we introduce the first comprehensive RNA benchmark BEACON (\textbf{BE}nchm\textbf{A}rk for \textbf{CO}mprehensive R\textbf{N}A Task and Language Models). First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications, enabling a comprehensive assessment of the performance of methods on various RNA understanding tasks. Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models. Third, we investigate the vital RNA language model components from the tokenizer and positional encoding aspects. Notably, our findings emphasize the superiority of single nucleotide tokenization and the effectiveness of Attention with Linear Biases (ALiBi) over traditional positional encoding methods. Based on these insights, a simple yet strong baseline called BEACON-B is proposed, which can achieve outstanding performance with limited data and computational resources. The datasets and source code of our benchmark are available at https://github.com/terry-r123/RNABenchmark. |
| title | BEACON: Benchmark for Comprehensive RNA Tasks and Language Models |
| topic | Quantitative Methods Machine Learning |
| url | https://arxiv.org/abs/2406.10391 |