BeamClean: Language Aware Embedding Reconstruction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913848021221376 |
|---|---|
| author | Kale, Kaan Mylonakis, Kyle Roberts, Jay Roy, Sidhartha |
| author_facet | Kale, Kaan Mylonakis, Kyle Roberts, Jay Roy, Sidhartha |
| contents | In this work, we consider an inversion attack on the obfuscated input embeddings sent to a language model on a server, where the adversary has no access to the language model or the obfuscation mechanism and sees only the obfuscated embeddings along with the model's embedding table. We propose BeamClean, an inversion attack that jointly estimates the noise parameters and decodes token sequences by integrating a language-model prior. Against Laplacian and Gaussian obfuscation mechanisms, BeamClean always surpasses naive distance-based attacks. This work highlights the necessity for and robustness of more advanced learned, input-dependent methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13758 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BeamClean: Language Aware Embedding Reconstruction Kale, Kaan Mylonakis, Kyle Roberts, Jay Roy, Sidhartha Cryptography and Security In this work, we consider an inversion attack on the obfuscated input embeddings sent to a language model on a server, where the adversary has no access to the language model or the obfuscation mechanism and sees only the obfuscated embeddings along with the model's embedding table. We propose BeamClean, an inversion attack that jointly estimates the noise parameters and decodes token sequences by integrating a language-model prior. Against Laplacian and Gaussian obfuscation mechanisms, BeamClean always surpasses naive distance-based attacks. This work highlights the necessity for and robustness of more advanced learned, input-dependent methods. |
| title | BeamClean: Language Aware Embedding Reconstruction |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2505.13758 |