Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based Strategies
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912042467721216 |
|---|---|
| author | Ye, Wenjin Ma, Yuanchen Xiang, Junkai Liang, Hongjie Wang, Tao Xiang, Qiuling Xiang, Andy Peng Song, Wu Li, Weiqiang Huang, Weijun |
| author_facet | Ye, Wenjin Ma, Yuanchen Xiang, Junkai Liang, Hongjie Wang, Tao Xiang, Qiuling Xiang, Andy Peng Song, Wu Li, Weiqiang Huang, Weijun |
| contents | Reliability in cell type annotation is challenging in single-cell RNA-sequencing data analysis because both expert-driven and automated methods can be biased or constrained by their training data, especially for novel or rare cell types. Although large language models (LLMs) are useful, our evaluation found that only a few matched expert annotations due to biased data sources and inflexible training inputs. To overcome these limitations, we developed the LICT (Large language model-based Identifier for Cell Types) software package using a multi-model fusion and "talk-to-machine" strategy. Tested across various single-cell RNA sequencing datasets, our approach significantly improved annotation reliability, especially in datasets with low cellular heterogeneity. Notably, we established objective criteria to assess annotation reliability using the "talk-to-machine" approach, which addresses discrepancies between our annotations and expert ones, enabling reliable evaluation even without reference data. This strategy enhances annotation credibility and sets the stage for advancing future LLM-based cell type annotation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15678 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based Strategies Ye, Wenjin Ma, Yuanchen Xiang, Junkai Liang, Hongjie Wang, Tao Xiang, Qiuling Xiang, Andy Peng Song, Wu Li, Weiqiang Huang, Weijun Quantitative Methods Genomics Reliability in cell type annotation is challenging in single-cell RNA-sequencing data analysis because both expert-driven and automated methods can be biased or constrained by their training data, especially for novel or rare cell types. Although large language models (LLMs) are useful, our evaluation found that only a few matched expert annotations due to biased data sources and inflexible training inputs. To overcome these limitations, we developed the LICT (Large language model-based Identifier for Cell Types) software package using a multi-model fusion and "talk-to-machine" strategy. Tested across various single-cell RNA sequencing datasets, our approach significantly improved annotation reliability, especially in datasets with low cellular heterogeneity. Notably, we established objective criteria to assess annotation reliability using the "talk-to-machine" approach, which addresses discrepancies between our annotations and expert ones, enabling reliable evaluation even without reference data. This strategy enhances annotation credibility and sets the stage for advancing future LLM-based cell type annotation methods. |
| title | Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based Strategies |
| topic | Quantitative Methods Genomics |
| url | https://arxiv.org/abs/2409.15678 |