SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910403341058048 |
|---|---|
| author | Rykov, Elisei Shishkina, Yana Petrushina, Kseniia Titova, Kseniia Petrakov, Sergey Panchenko, Alexander |
| author_facet | Rykov, Elisei Shishkina, Yana Petrushina, Kseniia Titova, Kseniia Petrakov, Sergey Panchenko, Alexander |
| contents | In this paper, we present our novel systems developed for the SemEval-2024 hallucination detection task. Our investigation spans a range of strategies to compare model predictions with reference standards, encompassing diverse baselines, the refinement of pre-trained encoders through supervised learning, and an ensemble approaches utilizing several high-performing models. Through these explorations, we introduce three distinct methods that exhibit strong performance metrics. To amplify our training data, we generate additional training samples from unlabelled training subset. Furthermore, we provide a detailed comparative analysis of our approaches. Notably, our premier method achieved a commendable 9th place in the competition's model-agnostic track and 17th place in model-aware track, highlighting its effectiveness and potential. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06137 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection Rykov, Elisei Shishkina, Yana Petrushina, Kseniia Titova, Kseniia Petrakov, Sergey Panchenko, Alexander Computation and Language Artificial Intelligence In this paper, we present our novel systems developed for the SemEval-2024 hallucination detection task. Our investigation spans a range of strategies to compare model predictions with reference standards, encompassing diverse baselines, the refinement of pre-trained encoders through supervised learning, and an ensemble approaches utilizing several high-performing models. Through these explorations, we introduce three distinct methods that exhibit strong performance metrics. To amplify our training data, we generate additional training samples from unlabelled training subset. Furthermore, we provide a detailed comparative analysis of our approaches. Notably, our premier method achieved a commendable 9th place in the competition's model-agnostic track and 17th place in model-aware track, highlighting its effectiveness and potential. |
| title | SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.06137 |