Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912849786306560 |
|---|---|
| author | Labroo, Arya Sheth, Ivaxi Raina, Vyas Ahmed, Amaani Fritz, Mario |
| author_facet | Labroo, Arya Sheth, Ivaxi Raina, Vyas Ahmed, Amaani Fritz, Mario |
| contents | Large Language Models (LLMs) offer strong generative capabilities, but many applications require explicit and \textit{fine-grained} control over specific textual concepts, such as humor, persuasiveness, or formality. Prior approaches in prompting and representation engineering can provide coarse or single-attribute control, but systematic evaluation of multi-attribute settings remains limited. We introduce an evaluation framework for fine-grained controllability for both single- and dual-concept scenarios, focusing on linguistically distinct concept pairs (e.g., persuasiveness vs.~humor). Surprisingly, across multiple LLMs and generative tasks, we find that performance often drops in the dual-concept setting, even though the chosen concepts should in principle be separable. This reveals a fundamental limitation of naive prompting-based control: models struggle with compositionality even when concepts are intuitively independent. Our framework provides systematic evidence of this gap and offers a principled approach for measuring the ability of future methods for multi-concept control. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18483 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs Labroo, Arya Sheth, Ivaxi Raina, Vyas Ahmed, Amaani Fritz, Mario Computation and Language Artificial Intelligence Large Language Models (LLMs) offer strong generative capabilities, but many applications require explicit and \textit{fine-grained} control over specific textual concepts, such as humor, persuasiveness, or formality. Prior approaches in prompting and representation engineering can provide coarse or single-attribute control, but systematic evaluation of multi-attribute settings remains limited. We introduce an evaluation framework for fine-grained controllability for both single- and dual-concept scenarios, focusing on linguistically distinct concept pairs (e.g., persuasiveness vs.~humor). Surprisingly, across multiple LLMs and generative tasks, we find that performance often drops in the dual-concept setting, even though the chosen concepts should in principle be separable. This reveals a fundamental limitation of naive prompting-based control: models struggle with compositionality even when concepts are intuitively independent. Our framework provides systematic evidence of this gap and offers a principled approach for measuring the ability of future methods for multi-concept control. |
| title | Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.18483 |