Rethinking Text-based Protein Understanding: Retrieval or LLM?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915608019337216 |
|---|---|
| author | Wu, Juntong Liu, Zijing Cao, He Li, Hao Feng, Bin Shu, Zishan Yu, Ke Yuan, Li Li, Yu |
| author_facet | Wu, Juntong Liu, Zijing Cao, He Li, Hao Feng, Bin Shu, Zishan Yu, Ke Yuan, Li Li, Yu |
| contents | In recent years, protein-text models have gained significant attention for their potential in protein generation and understanding. Current approaches focus on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment, enabling simultaneous comprehension of textual descriptions and protein sequences. Through a thorough analysis of existing model architectures and text-based protein understanding benchmarks, we identify significant data leakage issues present in current benchmarks. Moreover, conventional metrics derived from natural language processing fail to accurately assess the model's performance in this domain. To address these limitations, we reorganize existing datasets and introduce a novel evaluation framework based on biological entities. Motivated by our observation, we propose a retrieval-enhanced method, which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. Our code and data can be seen at https://github.com/IDEA-XL/RAPM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20354 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Text-based Protein Understanding: Retrieval or LLM? Wu, Juntong Liu, Zijing Cao, He Li, Hao Feng, Bin Shu, Zishan Yu, Ke Yuan, Li Li, Yu Computation and Language Artificial Intelligence In recent years, protein-text models have gained significant attention for their potential in protein generation and understanding. Current approaches focus on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment, enabling simultaneous comprehension of textual descriptions and protein sequences. Through a thorough analysis of existing model architectures and text-based protein understanding benchmarks, we identify significant data leakage issues present in current benchmarks. Moreover, conventional metrics derived from natural language processing fail to accurately assess the model's performance in this domain. To address these limitations, we reorganize existing datasets and introduce a novel evaluation framework based on biological entities. Motivated by our observation, we propose a retrieval-enhanced method, which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. Our code and data can be seen at https://github.com/IDEA-XL/RAPM. |
| title | Rethinking Text-based Protein Understanding: Retrieval or LLM? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.20354 |