Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.13874 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912819155304448 |
|---|---|
| author | Bystroński, Mateusz Han, Doheon Chawla, Nitesh V. Kajdanowicz, Tomasz |
| author_facet | Bystroński, Mateusz Han, Doheon Chawla, Nitesh V. Kajdanowicz, Tomasz |
| contents | Starting from the hypothesis that knowledge in semantic space is organized along structured manifolds, we argue that this geometric structure renders the space explorable. By traversing it and using the resulting continuous representations to condition an LLM's generation distribution, we can systematically expand the model's reachable semantic range. We introduce a framework that requires no modification of LLM parameters and operationalizes this idea by constructing a conditioning distribution from a small set of diverse anchor generations. This distribution conditions LLM's generation via an xRAG-style projector. Our experiments demonstrate that this manifold-based conditioning substantially increases generative diversity, with direct benefits for enhancing divergent thinking, a core facet of creativity, in language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_13874 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Models Bystroński, Mateusz Han, Doheon Chawla, Nitesh V. Kajdanowicz, Tomasz Artificial Intelligence Starting from the hypothesis that knowledge in semantic space is organized along structured manifolds, we argue that this geometric structure renders the space explorable. By traversing it and using the resulting continuous representations to condition an LLM's generation distribution, we can systematically expand the model's reachable semantic range. We introduce a framework that requires no modification of LLM parameters and operationalizes this idea by constructing a conditioning distribution from a small set of diverse anchor generations. This distribution conditions LLM's generation via an xRAG-style projector. Our experiments demonstrate that this manifold-based conditioning substantially increases generative diversity, with direct benefits for enhancing divergent thinking, a core facet of creativity, in language models. |
| title | Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2507.13874 |