Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.11575 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914260171358208 |
|---|---|
| author | Chytas, Sotirios Panagiotis Singh, Vikas |
| author_facet | Chytas, Sotirios Panagiotis Singh, Vikas |
| contents | Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these Attractors to solve a wide range of practical tasks, including language translation, hallucination reduction, guardrailing, and synthetic data generation. Despite their simplicity, these Attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_11575 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Concept Attractors in LLMs and their Applications Chytas, Sotirios Panagiotis Singh, Vikas Computation and Language Artificial Intelligence Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these Attractors to solve a wide range of practical tasks, including language translation, hallucination reduction, guardrailing, and synthetic data generation. Despite their simplicity, these Attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform. |
| title | Concept Attractors in LLMs and their Applications |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.11575 |