Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.12324 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912222777704448 |
|---|---|
| author | Weinberger, Edward D. |
| author_facet | Weinberger, Edward D. |
| contents | Standard information theory says nothing about how much meaning is conveyed by a message. We fill this gap with a rigorously justifiable, quantitative definition of ``pragmatic information'', the amount of meaning in a message relevant to a particular decision. We posit that such a message updates a random variable, $ω$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the prior and posterior probabilities of $ω$; the pragmatic information of a message ensemble is the expected value of the pragmatic information of the ensemble's component messages. We justify these definitions by proving that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $ω$ under the a priori and a posteriori distributions, and that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value.
Pragmatic information is non-negative and additive for independent decisions and ``pragmatically independent'' messages. Also, pragmatic information is the information analogue of free energy: just as free energy quantifies the part of a system's total energy available to do useful work, so pragmatic information quantifies the information actually used in making a decision.
We sketch 3 applications: the single play of a slot machine, a.k.a. a ``one armed bandit'', with an unknown payout probability; a characterization of the rate of biological evolution in the so-called ``quasi-species'' model; and a reformulation of the efficient market hypothesis of finance. We note the importance of the computational capacity of the receiver in each case. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12324 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards a Theory of Pragmatic Information Weinberger, Edward D. Information Theory H.1.1 Standard information theory says nothing about how much meaning is conveyed by a message. We fill this gap with a rigorously justifiable, quantitative definition of ``pragmatic information'', the amount of meaning in a message relevant to a particular decision. We posit that such a message updates a random variable, $ω$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the prior and posterior probabilities of $ω$; the pragmatic information of a message ensemble is the expected value of the pragmatic information of the ensemble's component messages. We justify these definitions by proving that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $ω$ under the a priori and a posteriori distributions, and that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value. Pragmatic information is non-negative and additive for independent decisions and ``pragmatically independent'' messages. Also, pragmatic information is the information analogue of free energy: just as free energy quantifies the part of a system's total energy available to do useful work, so pragmatic information quantifies the information actually used in making a decision. We sketch 3 applications: the single play of a slot machine, a.k.a. a ``one armed bandit'', with an unknown payout probability; a characterization of the rate of biological evolution in the so-called ``quasi-species'' model; and a reformulation of the efficient market hypothesis of finance. We note the importance of the computational capacity of the receiver in each case. |
| title | Towards a Theory of Pragmatic Information |
| topic | Information Theory H.1.1 |
| url | https://arxiv.org/abs/2403.12324 |