Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.15268 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913666584018944 |
|---|---|
| author | Qiang, Jipeng Huang, Minjiang Zhu, Yi Yuan, Yunhao Zhang, Chaowei Ouyang, Xiaoye |
| author_facet | Qiang, Jipeng Huang, Minjiang Zhu, Yi Yuan, Yunhao Zhang, Chaowei Ouyang, Xiaoye |
| contents | Lexical Simplification (LS) methods use a three-step pipeline: complex word identification, substitute generation, and substitute ranking, each with separate evaluation datasets. We found large language models (LLMs) can simplify sentences directly with a single prompt, bypassing the traditional pipeline. However, existing LS datasets are not suitable for evaluating these LLM-generated simplified sentences, as they focus on providing substitutes for single complex words without identifying all complex words in a sentence.
To address this gap, we propose a new annotation method for constructing an all-in-one LS dataset through human-machine collaboration. Automated methods generate a pool of potential substitutes, which human annotators then assess, suggesting additional alternatives as needed. Additionally, we explore LLM-based methods with single prompts, in-context learning, and chain-of-thought techniques. We introduce a multi-LLMs collaboration approach to simulate each step of the LS task. Experimental results demonstrate that LS based on multi-LLMs approaches significantly outperforms existing baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15268 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | New Evaluation Paradigm for Lexical Simplification Qiang, Jipeng Huang, Minjiang Zhu, Yi Yuan, Yunhao Zhang, Chaowei Ouyang, Xiaoye Computation and Language Lexical Simplification (LS) methods use a three-step pipeline: complex word identification, substitute generation, and substitute ranking, each with separate evaluation datasets. We found large language models (LLMs) can simplify sentences directly with a single prompt, bypassing the traditional pipeline. However, existing LS datasets are not suitable for evaluating these LLM-generated simplified sentences, as they focus on providing substitutes for single complex words without identifying all complex words in a sentence. To address this gap, we propose a new annotation method for constructing an all-in-one LS dataset through human-machine collaboration. Automated methods generate a pool of potential substitutes, which human annotators then assess, suggesting additional alternatives as needed. Additionally, we explore LLM-based methods with single prompts, in-context learning, and chain-of-thought techniques. We introduce a multi-LLMs collaboration approach to simulate each step of the LS task. Experimental results demonstrate that LS based on multi-LLMs approaches significantly outperforms existing baselines. |
| title | New Evaluation Paradigm for Lexical Simplification |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2501.15268 |