Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.11182 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915197852057600 |
|---|---|
| author | Yang, Zhe Huang, Yi Chen, Yaqin Wu, Xiaoting Feng, Junlan Deng, Chao |
| author_facet | Yang, Zhe Huang, Yi Chen, Yaqin Wu, Xiaoting Feng, Junlan Deng, Chao |
| contents | Recent advancements in large language models have revolutionized text generation with their remarkable capabilities. These models can produce controlled texts that closely adhere to specific requirements when prompted appropriately. However, designing an optimal prompt to control multiple attributes simultaneously can be challenging. A common approach is to linearly combine single-attribute models, but this strategy often overlooks attribute overlaps and can lead to conflicts. Therefore, we propose a novel combination strategy inspired by the Law of Total Probability and Conditional Mutual Information Minimization on generative language models. This method has been adapted for single-attribute control scenario and is termed the Palette of Language Models due to its theoretical linkage between attribute strength and generation style, akin to blending colors on an artist's palette. Moreover, positive correlation and attribute enhancement are advanced as theoretical properties to guide a rational combination strategy design. We conduct experiments on both single control and multiple control settings, and achieve surpassing results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11182 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Palette of Language Models: A Solver for Controlled Text Generation Yang, Zhe Huang, Yi Chen, Yaqin Wu, Xiaoting Feng, Junlan Deng, Chao Computation and Language Recent advancements in large language models have revolutionized text generation with their remarkable capabilities. These models can produce controlled texts that closely adhere to specific requirements when prompted appropriately. However, designing an optimal prompt to control multiple attributes simultaneously can be challenging. A common approach is to linearly combine single-attribute models, but this strategy often overlooks attribute overlaps and can lead to conflicts. Therefore, we propose a novel combination strategy inspired by the Law of Total Probability and Conditional Mutual Information Minimization on generative language models. This method has been adapted for single-attribute control scenario and is termed the Palette of Language Models due to its theoretical linkage between attribute strength and generation style, akin to blending colors on an artist's palette. Moreover, positive correlation and attribute enhancement are advanced as theoretical properties to guide a rational combination strategy design. We conduct experiments on both single control and multiple control settings, and achieve surpassing results. |
| title | Palette of Language Models: A Solver for Controlled Text Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.11182 |