Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.08389 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910646999711744 |
|---|---|
| author | Sun, Huashan Wu, Yixiao Ye, Yuhao Yang, Yizhe Li, Yinghao Li, Jiawei Gao, Yang |
| author_facet | Sun, Huashan Wu, Yixiao Ye, Yuhao Yang, Yizhe Li, Yinghao Li, Jiawei Gao, Yang |
| contents | Language style is necessary for AI systems to understand and generate diverse human language accurately. However, previous text style transfer primarily focused on sentence-level data-driven approaches, limiting exploration of potential problems in large language models (LLMs) and the ability to meet complex application needs. To overcome these limitations, we introduce a novel task called Public-Speaking Style Transfer (PSST), which aims to simulate humans to transform passage-level, official texts into a public-speaking style. Grounded in the analysis of real-world data from a linguistic perspective, we decompose public-speaking style into key sub-styles to pose challenges and quantify the style modeling capability of LLMs. For such intricate text style transfer, we further propose a fine-grained evaluation framework to analyze the characteristics and identify the problems of stylized texts. Comprehensive experiments suggest that current LLMs struggle to generate public speaking texts that align with human preferences, primarily due to excessive stylization and loss of semantic information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_08389 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PSST: A Benchmark for Evaluation-driven Text Public-Speaking Style Transfer Sun, Huashan Wu, Yixiao Ye, Yuhao Yang, Yizhe Li, Yinghao Li, Jiawei Gao, Yang Computation and Language Artificial Intelligence I.2.7 Language style is necessary for AI systems to understand and generate diverse human language accurately. However, previous text style transfer primarily focused on sentence-level data-driven approaches, limiting exploration of potential problems in large language models (LLMs) and the ability to meet complex application needs. To overcome these limitations, we introduce a novel task called Public-Speaking Style Transfer (PSST), which aims to simulate humans to transform passage-level, official texts into a public-speaking style. Grounded in the analysis of real-world data from a linguistic perspective, we decompose public-speaking style into key sub-styles to pose challenges and quantify the style modeling capability of LLMs. For such intricate text style transfer, we further propose a fine-grained evaluation framework to analyze the characteristics and identify the problems of stylized texts. Comprehensive experiments suggest that current LLMs struggle to generate public speaking texts that align with human preferences, primarily due to excessive stylization and loss of semantic information. |
| title | PSST: A Benchmark for Evaluation-driven Text Public-Speaking Style Transfer |
| topic | Computation and Language Artificial Intelligence I.2.7 |
| url | https://arxiv.org/abs/2311.08389 |