PERSOMA: PERsonalized SOft ProMpt Adapter Architecture for Personalized Language Prompting
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910552080515072 |
|---|---|
| author | Hebert, Liam Sayana, Krishna Jash, Ambarish Karatzoglou, Alexandros Sodhi, Sukhdeep Doddapaneni, Sumanth Cai, Yanli Kuzmin, Dima |
| author_facet | Hebert, Liam Sayana, Krishna Jash, Ambarish Karatzoglou, Alexandros Sodhi, Sukhdeep Doddapaneni, Sumanth Cai, Yanli Kuzmin, Dima |
| contents | Understanding the nuances of a user's extensive interaction history is key to building accurate and personalized natural language systems that can adapt to evolving user preferences. To address this, we introduce PERSOMA, Personalized Soft Prompt Adapter architecture. Unlike previous personalized prompting methods for large language models, PERSOMA offers a novel approach to efficiently capture user history. It achieves this by resampling and compressing interactions as free form text into expressive soft prompt embeddings, building upon recent research utilizing embedding representations as input for LLMs. We rigorously validate our approach by evaluating various adapter architectures, first-stage sampling strategies, parameter-efficient tuning techniques like LoRA, and other personalization methods. Our results demonstrate PERSOMA's superior ability to handle large and complex user histories compared to existing embedding-based and text-prompt-based techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00960 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PERSOMA: PERsonalized SOft ProMpt Adapter Architecture for Personalized Language Prompting Hebert, Liam Sayana, Krishna Jash, Ambarish Karatzoglou, Alexandros Sodhi, Sukhdeep Doddapaneni, Sumanth Cai, Yanli Kuzmin, Dima Computation and Language Artificial Intelligence Information Retrieval Understanding the nuances of a user's extensive interaction history is key to building accurate and personalized natural language systems that can adapt to evolving user preferences. To address this, we introduce PERSOMA, Personalized Soft Prompt Adapter architecture. Unlike previous personalized prompting methods for large language models, PERSOMA offers a novel approach to efficiently capture user history. It achieves this by resampling and compressing interactions as free form text into expressive soft prompt embeddings, building upon recent research utilizing embedding representations as input for LLMs. We rigorously validate our approach by evaluating various adapter architectures, first-stage sampling strategies, parameter-efficient tuning techniques like LoRA, and other personalization methods. Our results demonstrate PERSOMA's superior ability to handle large and complex user histories compared to existing embedding-based and text-prompt-based techniques. |
| title | PERSOMA: PERsonalized SOft ProMpt Adapter Architecture for Personalized Language Prompting |
| topic | Computation and Language Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2408.00960 |