Copertina

Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Bai, Bo
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Information Theory Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Towards a Measure Theory of Semantic Information
di: Coghill, George M.
Pubblicazione: (2025)

Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting
di: Cheng, Ran
Pubblicazione: (2026)

Toward a Theory of Hierarchical Memory for Language Agents
di: Talebirad, Yashar, et al.
Pubblicazione: (2026)

Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels
di: Wu, Tong, et al.
Pubblicazione: (2024)

Semantic Communication for Cooperative Perception using HARQ
di: Sheng, Yucheng, et al.
Pubblicazione: (2024)

On the Rate-Distortion-Complexity Tradeoff for Semantic Communication
di: Chai, Jingxuan, et al.
Pubblicazione: (2026)

Semantic Error Correction and Decoding for Short Block Codes
di: Hao, Jiafu, et al.
Pubblicazione: (2026)

Learning is Forgetting: LLM Training As Lossy Compression
di: Conklin, Henry C., et al.
Pubblicazione: (2026)

A Rational Account of Categorization Based on Information Theory
di: MacLellan, Christopher J., et al.
Pubblicazione: (2026)

A Mathematical Theory of Agency and Intelligence
di: Hafez, Wael, et al.
Pubblicazione: (2026)

Federated Latent Space Alignment for Multi-user Semantic Communications
di: Di Poce, Giuseppe, et al.
Pubblicazione: (2026)

On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training
di: Niu, Xueyan, et al.
Pubblicazione: (2026)

Random Walk in Random Permutation Set Theory
di: Zhou, Jiefeng, et al.
Pubblicazione: (2024)

Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
di: Xu, Shizhou, et al.
Pubblicazione: (2025)

Information-Theoretic Equivalence of Entropic Multi-Marginal Optimal Transport: A Theory for Multi-Agent Communication
di: Wang, Shuchan
Pubblicazione: (2022)

User-Intent-Driven Semantic Communication via Adaptive Deep Understanding
di: Ye, Peigen, et al.
Pubblicazione: (2025)

Generalization Bounds: Perspectives from Information Theory and PAC-Bayes
di: Hellström, Fredrik, et al.
Pubblicazione: (2023)

Solomonoff-Inspired Hypothesis Ranking with LLMs for Prediction Under Uncertainty
di: Barber, Josh, et al.
Pubblicazione: (2025)

All Substitution Is Local
di: Shah, Nidhish, et al.
Pubblicazione: (2026)

Context Video Semantic Transmission with Variable Length and Rate Coding over MIMO Channels
di: Xie, Bingyan, et al.
Pubblicazione: (2025)

WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference
di: Liu, Zixuan, et al.
Pubblicazione: (2026)

Rateless Joint Source-Channel Coding, and a Blueprint for 6G Semantic Communications System Design
di: Khosravirad, Saeed R.
Pubblicazione: (2025)

Deep DNA Storage: Scalable and Robust DNA Storage via Coding Theory and Deep Learning
di: Bar-Lev, Daniella, et al.
Pubblicazione: (2021)

Intensional Inheritance Between Concepts: An Information-Theoretic Interpretation
di: Goertzel, Ben
Pubblicazione: (2025)

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization
di: Shwartz-Ziv, Ravid, et al.
Pubblicazione: (2023)

Treatment of Epistemic Uncertainty in Conjunction Analysis with Dempster-Shafer Theory
di: Sanchez, Luis, et al.
Pubblicazione: (2024)

Computation Rate Maximization for Wireless Powered Edge Computing With Multi-User Cooperation
di: Li, Yang, et al.
Pubblicazione: (2024)

Co-learning-aided Multi-modal-deep-learning Framework of Passive DOA Estimators for a Heterogeneous Hybrid Massive MIMO Receiver
di: Bai, Jiatong, et al.
Pubblicazione: (2024)

Semantic Channel Theory: Deductive Compression and Structural Fidelity for Multi-Agent Communication
di: Xu, Jianfeng
Pubblicazione: (2026)

How to Evaluate Semantic Communications for Images with ViTScore Metric?
di: Zhu, Tingting, et al.
Pubblicazione: (2023)

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
di: Anwar, Usman, et al.
Pubblicazione: (2026)

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?
di: Català, Mar Gonzàlez I, et al.
Pubblicazione: (2026)

MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
di: Bhattacharjee, Payel, et al.
Pubblicazione: (2026)

Discrete-Space Generative AI Pipeline for Semantic Transmission of Signals
di: Kokalj-Filipovic, Silvija, et al.
Pubblicazione: (2026)

Knowledge Graph-Based Explainable and Generalized Zero-Shot Semantic Communications
di: Zhang, Zhaoyu, et al.
Pubblicazione: (2025)

Contextuality as an External Bookkeeping Cost under Fixed Shared-State Semantics
di: Kim, Song-Ju
Pubblicazione: (2026)

Benchmarking Semantic Communications for Image Transmission Over MIMO Interference Channels
di: Wang, Yanhu, et al.
Pubblicazione: (2024)

NestQuant: Nested Lattice Quantization for Matrix Products and LLMs
di: Savkin, Semyon, et al.
Pubblicazione: (2025)

Balancing Information Accuracy and Response Timeliness in Networked LLMs
di: Turkmen, Yigit, et al.
Pubblicazione: (2025)

Semantic Rate Distortion and Posterior Design: Compute Constraints, Multimodality, and Strategic Inference
di: Akyol, Emrah
Pubblicazione: (2026)