Auditing Prompt Caching in Language Model APIs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915384811061248 |
|---|---|
| author | Gu, Chenchen Li, Xiang Lisa Kuditipudi, Rohith Liang, Percy Hashimoto, Tatsunori |
| author_facet | Gu, Chenchen Li, Xiang Lisa Kuditipudi, Rohith Liang, Percy Hashimoto, Tatsunori |
| contents | Prompt caching in large language models (LLMs) results in data-dependent timing variations: cached prompts are processed faster than non-cached prompts. These timing differences introduce the risk of side-channel timing attacks. For example, if the cache is shared across users, an attacker could identify cached prompts from fast API response times to learn information about other users' prompts. Because prompt caching may cause privacy leakage, transparency around the caching policies of API providers is important. To this end, we develop and conduct statistical audits to detect prompt caching in real-world LLM API providers. We detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts. Timing variations due to prompt caching can also result in leakage of information about model architecture. Namely, we find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_07776 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Auditing Prompt Caching in Language Model APIs Gu, Chenchen Li, Xiang Lisa Kuditipudi, Rohith Liang, Percy Hashimoto, Tatsunori Computation and Language Cryptography and Security Machine Learning Prompt caching in large language models (LLMs) results in data-dependent timing variations: cached prompts are processed faster than non-cached prompts. These timing differences introduce the risk of side-channel timing attacks. For example, if the cache is shared across users, an attacker could identify cached prompts from fast API response times to learn information about other users' prompts. Because prompt caching may cause privacy leakage, transparency around the caching policies of API providers is important. To this end, we develop and conduct statistical audits to detect prompt caching in real-world LLM API providers. We detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts. Timing variations due to prompt caching can also result in leakage of information about model architecture. Namely, we find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known. |
| title | Auditing Prompt Caching in Language Model APIs |
| topic | Computation and Language Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2502.07776 |