LUNA: A Framework for Language Understanding and Naturalness Assessment
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916085648850944 |
|---|---|
| author | Saidov, Marat Bakalova, Aleksandra Taktasheva, Ekaterina Mikhailov, Vladislav Artemova, Ekaterina |
| author_facet | Saidov, Marat Bakalova, Aleksandra Taktasheva, Ekaterina Mikhailov, Vladislav Artemova, Ekaterina |
| contents | The evaluation of Natural Language Generation (NLG) models has gained increased attention, urging the development of metrics that evaluate various aspects of generated text. LUNA addresses this challenge by introducing a unified interface for 20 NLG evaluation metrics. These metrics are categorized based on their reference-dependence and the type of text representation they employ, from string-based n-gram overlap to the utilization of static embeddings and pre-trained language models.
The straightforward design of LUNA allows for easy extension with novel metrics, requiring just a few lines of code. LUNA offers a user-friendly tool for evaluating generated texts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04522 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LUNA: A Framework for Language Understanding and Naturalness Assessment Saidov, Marat Bakalova, Aleksandra Taktasheva, Ekaterina Mikhailov, Vladislav Artemova, Ekaterina Computation and Language The evaluation of Natural Language Generation (NLG) models has gained increased attention, urging the development of metrics that evaluate various aspects of generated text. LUNA addresses this challenge by introducing a unified interface for 20 NLG evaluation metrics. These metrics are categorized based on their reference-dependence and the type of text representation they employ, from string-based n-gram overlap to the utilization of static embeddings and pre-trained language models. The straightforward design of LUNA allows for easy extension with novel metrics, requiring just a few lines of code. LUNA offers a user-friendly tool for evaluating generated texts. |
| title | LUNA: A Framework for Language Understanding and Naturalness Assessment |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.04522 |