Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913308509995008 |
|---|---|
| author | Anschütz, Miriam Mosca, Edoardo Groh, Georg |
| author_facet | Anschütz, Miriam Mosca, Edoardo Groh, Georg |
| contents | Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionally, we conduct a detailed analysis of the correlation between prediction change rates and simplification types/strengths. Our findings reveal alarming inconsistencies across all languages and models. If not promptly addressed, simplified inputs can be easily exploited to craft zero-iteration model-agnostic adversarial attacks with success rates of up to 50% |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora? Anschütz, Miriam Mosca, Edoardo Groh, Georg Computation and Language Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionally, we conduct a detailed analysis of the correlation between prediction change rates and simplification types/strengths. Our findings reveal alarming inconsistencies across all languages and models. If not promptly addressed, simplified inputs can be easily exploited to craft zero-iteration model-agnostic adversarial attacks with success rates of up to 50% |
| title | Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.06838 |