Step-Back Profiling: Distilling User History for Personalized Scientific Writing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911951922135040 |
|---|---|
| author | Tang, Xiangru Zhang, Xingyao Shao, Yanjun Wu, Jie Zhao, Yilun Cohan, Arman Gong, Ming Zhang, Dongmei Gerstein, Mark |
| author_facet | Tang, Xiangru Zhang, Xingyao Shao, Yanjun Wu, Jie Zhao, Yilun Cohan, Arman Gong, Ming Zhang, Dongmei Gerstein, Mark |
| contents | Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals, particularly in real-world scenarios like scientific writing. Addressing this challenge, we introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles, including essential traits and preferences of users. To conduct the experiments, we construct a Personalized Scientific Writing (PSW) dataset to study multi-user personalization. PSW requires the models to write scientific papers given specialized author groups with diverse academic backgrounds. As for the results, we demonstrate the effectiveness of capturing user characteristics via STEP-BACK PROFILING for collaborative writing. Moreover, our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark (LaMP), including 7 personalization LLM tasks. Our ablation studies validate the contributions of different components in our method and provide insights into our task definition. Our dataset and code are available at \url{https://github.com/gersteinlab/step-back-profiling}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14275 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Step-Back Profiling: Distilling User History for Personalized Scientific Writing Tang, Xiangru Zhang, Xingyao Shao, Yanjun Wu, Jie Zhao, Yilun Cohan, Arman Gong, Ming Zhang, Dongmei Gerstein, Mark Computation and Language Artificial Intelligence Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals, particularly in real-world scenarios like scientific writing. Addressing this challenge, we introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles, including essential traits and preferences of users. To conduct the experiments, we construct a Personalized Scientific Writing (PSW) dataset to study multi-user personalization. PSW requires the models to write scientific papers given specialized author groups with diverse academic backgrounds. As for the results, we demonstrate the effectiveness of capturing user characteristics via STEP-BACK PROFILING for collaborative writing. Moreover, our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark (LaMP), including 7 personalization LLM tasks. Our ablation studies validate the contributions of different components in our method and provide insights into our task definition. Our dataset and code are available at \url{https://github.com/gersteinlab/step-back-profiling}. |
| title | Step-Back Profiling: Distilling User History for Personalized Scientific Writing |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.14275 |