Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.11736 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910703510618112 |
|---|---|
| author | Gritsai, German Voznyuk, Anastasia Khabutdinov, Ildar Grabovoy, Andrey |
| author_facet | Gritsai, German Voznyuk, Anastasia Khabutdinov, Ildar Grabovoy, Andrey |
| contents | The paper describes a system designed by Advacheck team to recognise machine-generated and human-written texts in the monolingual subtask of GenAI Detection Task 1 competition. Our developed system is a multi-task architecture with shared Transformer Encoder between several classification heads. One head is responsible for binary classification between human-written and machine-generated texts, while the other heads are auxiliary multiclass classifiers for texts of different domains from particular datasets. As multiclass heads were trained to distinguish the domains presented in the data, they provide a better understanding of the samples. This approach led us to achieve the first place in the official ranking with 83.07% macro F1-score on the test set and bypass the baseline by 10%. We further study obtained system through ablation, error and representation analyses, finding that multi-task learning outperforms single-task mode and simultaneous tasks form a cluster structure in embeddings space. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11736 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking Gritsai, German Voznyuk, Anastasia Khabutdinov, Ildar Grabovoy, Andrey Computation and Language The paper describes a system designed by Advacheck team to recognise machine-generated and human-written texts in the monolingual subtask of GenAI Detection Task 1 competition. Our developed system is a multi-task architecture with shared Transformer Encoder between several classification heads. One head is responsible for binary classification between human-written and machine-generated texts, while the other heads are auxiliary multiclass classifiers for texts of different domains from particular datasets. As multiclass heads were trained to distinguish the domains presented in the data, they provide a better understanding of the samples. This approach led us to achieve the first place in the official ranking with 83.07% macro F1-score on the test set and bypass the baseline by 10%. We further study obtained system through ablation, error and representation analyses, finding that multi-task learning outperforms single-task mode and simultaneous tasks form a cluster structure in embeddings space. |
| title | Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.11736 |