An Embarrassingly Simple Approach to Enhance Transformer Performance in Genomic Selection for Crop Breeding
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916298550673408 |
|---|---|
| author | Chen, Renqi Han, Wenwei Zhang, Haohao Su, Haoyang Wang, Zhefan Liu, Xiaolei Jiang, Hao Ouyang, Wanli Dong, Nanqing |
| author_facet | Chen, Renqi Han, Wenwei Zhang, Haohao Su, Haoyang Wang, Zhefan Liu, Xiaolei Jiang, Hao Ouyang, Wanli Dong, Nanqing |
| contents | Genomic selection (GS), as a critical crop breeding strategy, plays a key role in enhancing food production and addressing the global hunger crisis. The predominant approaches in GS currently revolve around employing statistical methods for prediction. However, statistical methods often come with two main limitations: strong statistical priors and linear assumptions. A recent trend is to capture the non-linear relationships between markers by deep learning. However, as crop datasets are commonly long sequences with limited samples, the robustness of deep learning models, especially Transformers, remains a challenge. In this work, to unleash the unexplored potential of attention mechanism for the task of interest, we propose a simple yet effective Transformer-based framework that enables end-to-end training of the whole sequence. Via experiments on rice3k and wheat3k datasets, we show that, with simple tricks such as k-mer tokenization and random masking, Transformer can achieve overall superior performance against seminal methods on GS tasks of interest. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_09585 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Embarrassingly Simple Approach to Enhance Transformer Performance in Genomic Selection for Crop Breeding Chen, Renqi Han, Wenwei Zhang, Haohao Su, Haoyang Wang, Zhefan Liu, Xiaolei Jiang, Hao Ouyang, Wanli Dong, Nanqing Machine Learning Artificial Intelligence Genomic selection (GS), as a critical crop breeding strategy, plays a key role in enhancing food production and addressing the global hunger crisis. The predominant approaches in GS currently revolve around employing statistical methods for prediction. However, statistical methods often come with two main limitations: strong statistical priors and linear assumptions. A recent trend is to capture the non-linear relationships between markers by deep learning. However, as crop datasets are commonly long sequences with limited samples, the robustness of deep learning models, especially Transformers, remains a challenge. In this work, to unleash the unexplored potential of attention mechanism for the task of interest, we propose a simple yet effective Transformer-based framework that enables end-to-end training of the whole sequence. Via experiments on rice3k and wheat3k datasets, we show that, with simple tricks such as k-mer tokenization and random masking, Transformer can achieve overall superior performance against seminal methods on GS tasks of interest. |
| title | An Embarrassingly Simple Approach to Enhance Transformer Performance in Genomic Selection for Crop Breeding |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2405.09585 |