Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.00612 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913632287195136 |
|---|---|
| author | Lastras, Luis A. Trager, Barry M. Lenchner, Jonathan Szpankowski, Wojciech Wu, Chai Wah Squillante, Mark S. Gray, Alexander |
| author_facet | Lastras, Luis A. Trager, Barry M. Lenchner, Jonathan Szpankowski, Wojciech Wu, Chai Wah Squillante, Mark S. Gray, Alexander |
| contents | Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the number of bits needed to encode and communicate a string of symbols. When he introduced the theory, Shannon famously excluded any notion of semantics behind the symbols being communicated. This semantics-free notion went on to have massive impact on communication and computing technologies, even as multiple proposals for reintroducing semantics in a theory of information were being made, notably one where Carnap and Bar-Hillel used logic and reasoning to capture semantics. In this paper we present, for the first time, a Shannon-style analysis of a communication system equipped with a deductive reasoning capability, implemented using logical inference. We use some of the most important techniques developed in information theory to demonstrate significant and sometimes surprising gains in communication efficiency availed to us through such capability, demonstrated also through practical codes. We thus argue that proposals for a semantic information theory should include the power of deductive reasoning to magnify the value of transmitted bits as we strive to fully unlock the inherent potential of semantics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00612 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Breaking through the classical Shannon entropy limit: A new frontier through logical semantics Lastras, Luis A. Trager, Barry M. Lenchner, Jonathan Szpankowski, Wojciech Wu, Chai Wah Squillante, Mark S. Gray, Alexander Information Theory Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the number of bits needed to encode and communicate a string of symbols. When he introduced the theory, Shannon famously excluded any notion of semantics behind the symbols being communicated. This semantics-free notion went on to have massive impact on communication and computing technologies, even as multiple proposals for reintroducing semantics in a theory of information were being made, notably one where Carnap and Bar-Hillel used logic and reasoning to capture semantics. In this paper we present, for the first time, a Shannon-style analysis of a communication system equipped with a deductive reasoning capability, implemented using logical inference. We use some of the most important techniques developed in information theory to demonstrate significant and sometimes surprising gains in communication efficiency availed to us through such capability, demonstrated also through practical codes. We thus argue that proposals for a semantic information theory should include the power of deductive reasoning to magnify the value of transmitted bits as we strive to fully unlock the inherent potential of semantics. |
| title | Breaking through the classical Shannon entropy limit: A new frontier through logical semantics |
| topic | Information Theory |
| url | https://arxiv.org/abs/2501.00612 |