The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914504073281536 |
|---|---|
| author | Fay, Aideen García-Redondo, Inés Wang, Qiquan Dubossarsky, Haim Monod, Anthea |
| author_facet | Fay, Aideen García-Redondo, Inés Wang, Qiquan Dubossarsky, Haim Monod, Anthea |
| contents | Existing interpretability methods for Large Language Models (LLMs) predominantly capture linear directions or isolated features. This overlooks the high-dimensional, relational, and nonlinear geometry of model representations. We apply persistent homology (PH) to characterize how adversarial inputs reshape the geometry and topology of internal representation spaces of LLMs. This phenomenon, especially when considered across operationally different attack modes, remains poorly understood. We analyze six models (3.8B to 70B parameters) under two distinct attacks, indirect prompt injection and backdoor fine--tuning, and show that a consistent topological signature persists throughout. Adversarial inputs induce topological compression, where the latent space becomes structurally simpler, collapsing the latent space from varied, compact, small-scale features into fewer, dominant, large-scale ones. This signature is architecture-agnostic, emerges early in the network, and is highly discriminative across layers. By quantifying the shape of activation point clouds and neuron-level information flow, our framework reveals geometric invariants of representational change that complement existing linear interpretability methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20435 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology Fay, Aideen García-Redondo, Inés Wang, Qiquan Dubossarsky, Haim Monod, Anthea Machine Learning Artificial Intelligence Computational Geometry Algebraic Topology Existing interpretability methods for Large Language Models (LLMs) predominantly capture linear directions or isolated features. This overlooks the high-dimensional, relational, and nonlinear geometry of model representations. We apply persistent homology (PH) to characterize how adversarial inputs reshape the geometry and topology of internal representation spaces of LLMs. This phenomenon, especially when considered across operationally different attack modes, remains poorly understood. We analyze six models (3.8B to 70B parameters) under two distinct attacks, indirect prompt injection and backdoor fine--tuning, and show that a consistent topological signature persists throughout. Adversarial inputs induce topological compression, where the latent space becomes structurally simpler, collapsing the latent space from varied, compact, small-scale features into fewer, dominant, large-scale ones. This signature is architecture-agnostic, emerges early in the network, and is highly discriminative across layers. By quantifying the shape of activation point clouds and neuron-level information flow, our framework reveals geometric invariants of representational change that complement existing linear interpretability methods. |
| title | The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology |
| topic | Machine Learning Artificial Intelligence Computational Geometry Algebraic Topology |
| url | https://arxiv.org/abs/2505.20435 |