A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918165849571328 |
|---|---|
| author | Van Huynh, Tin Van Nguyen, Kiet Nguyen, Ngan Luu-Thuy |
| author_facet | Van Huynh, Tin Van Nguyen, Kiet Nguyen, Ngan Luu-Thuy |
| contents | Existing Vietnamese Natural Language Inference (NLI) datasets lack adversarial complexity, limiting their ability to evaluate model robustness against challenging linguistic phenomena. In this article, we address the gap in robust Vietnamese NLI resources by introducing ViANLI, the first adversarial NLI dataset for Vietnamese, and propose NLIMoE, a Mixture-of-Experts model to tackle its complexity. We construct ViANLI using an adversarial human-and-machine-in-the-loop approach with rigorous verification. NLIMoE integrates expert subnetworks with a learned dynamic routing mechanism on top of a shared transformer encoder. ViANLI comprises over 10,000 premise-hypothesis pairs and challenges state-of-the-art models, with XLM-R Large achieving only 45.5% accuracy, while NLIMoE reaches 47.3%. Training with ViANLI improves performance on other benchmark Vietnamese NLI datasets including ViNLI, VLSP2021-NLI, and VnNewsNLI. ViANLI is released for enhancing research into model robustness and enriching resources for future Vietnamese and multilingual NLI research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_17716 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese Van Huynh, Tin Van Nguyen, Kiet Nguyen, Ngan Luu-Thuy Computation and Language Existing Vietnamese Natural Language Inference (NLI) datasets lack adversarial complexity, limiting their ability to evaluate model robustness against challenging linguistic phenomena. In this article, we address the gap in robust Vietnamese NLI resources by introducing ViANLI, the first adversarial NLI dataset for Vietnamese, and propose NLIMoE, a Mixture-of-Experts model to tackle its complexity. We construct ViANLI using an adversarial human-and-machine-in-the-loop approach with rigorous verification. NLIMoE integrates expert subnetworks with a learned dynamic routing mechanism on top of a shared transformer encoder. ViANLI comprises over 10,000 premise-hypothesis pairs and challenges state-of-the-art models, with XLM-R Large achieving only 45.5% accuracy, while NLIMoE reaches 47.3%. Training with ViANLI improves performance on other benchmark Vietnamese NLI datasets including ViNLI, VLSP2021-NLI, and VnNewsNLI. ViANLI is released for enhancing research into model robustness and enriching resources for future Vietnamese and multilingual NLI research. |
| title | A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamese |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.17716 |