Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.16653 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916222979801088 |
|---|---|
| author | Moraes, Lavínia de Carvalho Silvério, Irene Cristina Marques, Rafael Alexandre Sousa Anaia, Bianca de Castro de Paula, Dandara Freitas de Faria, Maria Carolina Schincariol Cleveston, Iury Correia, Alana de Santana Freitag, Raquel Meister Ko |
| author_facet | Moraes, Lavínia de Carvalho Silvério, Irene Cristina Marques, Rafael Alexandre Sousa Anaia, Bianca de Castro de Paula, Dandara Freitas de Faria, Maria Carolina Schincariol Cleveston, Iury Correia, Alana de Santana Freitag, Raquel Meister Ko |
| contents | Linguistic ambiguity continues to represent a significant challenge for natural language processing (NLP) systems, notwithstanding the advancements in architectures such as Transformers and BERT. Inspired by the recent success of instructional models like ChatGPT and Gemini (In 2023, the artificial intelligence was called Bard.), this study aims to analyze and discuss linguistic ambiguity within these models, focusing on three types prevalent in Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a corpus comprising 120 sentences, both ambiguous and unambiguous, for classification, explanation, and disambiguation. The models capability to generate ambiguous sentences was also explored by soliciting sets of sentences for each type of ambiguity. The results underwent qualitative analysis, drawing on recognized linguistic references, and quantitative assessment based on the accuracy of the responses obtained. It was evidenced that even the most sophisticated models, such as ChatGPT and Gemini, exhibit errors and deficiencies in their responses, with explanations often providing inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the need for descriptive studies for supervised learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_16653 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Análise de ambiguidade linguística em modelos de linguagem de grande escala (LLMs) Moraes, Lavínia de Carvalho Silvério, Irene Cristina Marques, Rafael Alexandre Sousa Anaia, Bianca de Castro de Paula, Dandara Freitas de Faria, Maria Carolina Schincariol Cleveston, Iury Correia, Alana de Santana Freitag, Raquel Meister Ko Computation and Language Artificial Intelligence Linguistic ambiguity continues to represent a significant challenge for natural language processing (NLP) systems, notwithstanding the advancements in architectures such as Transformers and BERT. Inspired by the recent success of instructional models like ChatGPT and Gemini (In 2023, the artificial intelligence was called Bard.), this study aims to analyze and discuss linguistic ambiguity within these models, focusing on three types prevalent in Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a corpus comprising 120 sentences, both ambiguous and unambiguous, for classification, explanation, and disambiguation. The models capability to generate ambiguous sentences was also explored by soliciting sets of sentences for each type of ambiguity. The results underwent qualitative analysis, drawing on recognized linguistic references, and quantitative assessment based on the accuracy of the responses obtained. It was evidenced that even the most sophisticated models, such as ChatGPT and Gemini, exhibit errors and deficiencies in their responses, with explanations often providing inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the need for descriptive studies for supervised learning. |
| title | Análise de ambiguidade linguística em modelos de linguagem de grande escala (LLMs) |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2404.16653 |