An Additive Approximation Scheme for Generating Dyadic Codings for the Outputs of an LLM
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918487102849024 |
|---|---|
| author | Bar-Lev, Daniella Farnoud, Farzad Gabrys, Ryan |
| author_facet | Bar-Lev, Daniella Farnoud, Farzad Gabrys, Ryan |
| contents | We study the problem of approximating a discrete probability distribution, such as the next-token distribution of a large language model, by a dyadic distribution induced by a binary tree under encoding rate constraints. The objective is to partition the support of the distribution and assign dyadic probabilities to minimize total variation distance while achieving a prescribed rate. We formulate this task as a tree-based partitioning problem and develop a polynomial-time additive approximation scheme for the rate-constrained setting in the constant-rate regime. Our results provide provable guarantees for near-optimal dyadic approximations and, as an application, yield a principled framework for LLM-based steganography, where the rate maps to bits of hidden information embedded per token and the total variation bound controls statistical detectability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_05837 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | An Additive Approximation Scheme for Generating Dyadic Codings for the Outputs of an LLM Bar-Lev, Daniella Farnoud, Farzad Gabrys, Ryan Information Theory Data Structures and Algorithms We study the problem of approximating a discrete probability distribution, such as the next-token distribution of a large language model, by a dyadic distribution induced by a binary tree under encoding rate constraints. The objective is to partition the support of the distribution and assign dyadic probabilities to minimize total variation distance while achieving a prescribed rate. We formulate this task as a tree-based partitioning problem and develop a polynomial-time additive approximation scheme for the rate-constrained setting in the constant-rate regime. Our results provide provable guarantees for near-optimal dyadic approximations and, as an application, yield a principled framework for LLM-based steganography, where the rate maps to bits of hidden information embedded per token and the total variation bound controls statistical detectability. |
| title | An Additive Approximation Scheme for Generating Dyadic Codings for the Outputs of an LLM |
| topic | Information Theory Data Structures and Algorithms |
| url | https://arxiv.org/abs/2605.05837 |