Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911042516287488 |
|---|---|
| author | Xiong, Chenfei Ni, Jingwei Fan, Yu Zouhar, Vilém Rooein, Donya Calvo-Bartolomé, Lorena Hoyle, Alexander Jin, Zhijing Sachan, Mrinmaya Leippold, Markus Hovy, Dirk El-Assady, Mennatallah Ash, Elliott |
| author_facet | Xiong, Chenfei Ni, Jingwei Fan, Yu Zouhar, Vilém Rooein, Donya Calvo-Bartolomé, Lorena Hoyle, Alexander Jin, Zhijing Sachan, Mrinmaya Leippold, Markus Hovy, Dirk El-Assady, Mennatallah Ash, Elliott |
| contents | We introduce Co-DETECT (Collaborative Discovery of Edge cases in TExt ClassificaTion), a novel mixed-initiative annotation framework that integrates human expertise with automatic annotation guided by large language models (LLMs). Co-DETECT starts with an initial, sketch-level codebook and dataset provided by a domain expert, then leverages the LLM to annotate the data and identify edge cases that are not well described by the initial codebook. Specifically, Co-DETECT flags challenging examples, induces high-level, generalizable descriptions of edge cases, and assists user in incorporating edge case handling rules to improve the codebook. This iterative process enables more effective handling of nuanced phenomena through compact, generalizable annotation rules. Extensive user study, qualitative and quantitative analyses prove the effectiveness of Co-DETECT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05010 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification Xiong, Chenfei Ni, Jingwei Fan, Yu Zouhar, Vilém Rooein, Donya Calvo-Bartolomé, Lorena Hoyle, Alexander Jin, Zhijing Sachan, Mrinmaya Leippold, Markus Hovy, Dirk El-Assady, Mennatallah Ash, Elliott Computation and Language We introduce Co-DETECT (Collaborative Discovery of Edge cases in TExt ClassificaTion), a novel mixed-initiative annotation framework that integrates human expertise with automatic annotation guided by large language models (LLMs). Co-DETECT starts with an initial, sketch-level codebook and dataset provided by a domain expert, then leverages the LLM to annotate the data and identify edge cases that are not well described by the initial codebook. Specifically, Co-DETECT flags challenging examples, induces high-level, generalizable descriptions of edge cases, and assists user in incorporating edge case handling rules to improve the codebook. This iterative process enables more effective handling of nuanced phenomena through compact, generalizable annotation rules. Extensive user study, qualitative and quantitative analyses prove the effectiveness of Co-DETECT. |
| title | Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.05010 |