Cover Image

A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Bouchaffra, Djamel
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence 68T07, 91A12, 94A17 G.3; H.1.1; I.2.6
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Compressive sensing inspired self-supervised single-pixel imaging
by: Lu, Jijun, et al.
Published: (2026)

Information-Theoretic Measures in AI: A Practical Decision Guide
by: Papadopoulos, Nikolaos Al., et al.
Published: (2026)

On the Optimal Representation Efficiency of Barlow Twins: An Information-Geometric Interpretation
by: Zhang, Di
Published: (2025)

Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence
by: Semenenko, Alexander, et al.
Published: (2025)

Bandwidth-Efficient Multi-Agent Communication through Information Bottleneck and Vector Quantization
by: Farooq, Ahmad, et al.
Published: (2026)

Semantic Variational Bayes Based on Semantic Information G Theory for Solving Latent Variables
by: Lu, Chenguang
Published: (2024)

Momentum Attention: The Physics of In-Context Learning and Spectral Forensics for Mechanistic Interpretability
by: Maitra, Kingsuk
Published: (2026)

Nyström $M$-Hilbert-Schmidt Independence Criterion
by: Kalinke, Florian, et al.
Published: (2023)

Machine learning based animal emotion classification using audio signals
by: Slobodian, Mariia, et al.
Published: (2025)

How Does Machine Learning Manage Complexity?
by: Fortnow, Lance
Published: (2026)

A Semantic Generalization of Shannon's Information Theory and Applications
by: Lu, Chenguang
Published: (2025)

Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories
by: Alpay, Faruk
Published: (2025)

HELIX: Scaling Raw Audio Understanding with Hybrid Mamba-Attention Beyond the Quadratic Limit
by: Khushiyant, et al.
Published: (2026)

FMMI: Flow Matching Mutual Information Estimation
by: Butakov, Ivan, et al.
Published: (2025)

Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression
by: Butakov, Ivan, et al.
Published: (2023)

Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax
by: Butakov, Ivan, et al.
Published: (2024)

Laws of Learning Dynamics and the Core of Learners
by: Jung, Inkee, et al.
Published: (2026)

The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence
by: Cai, Yichao, et al.
Published: (2026)

A Family of LZ78-based Universal Sequential Probability Assignments
by: Sagan, Naomi, et al.
Published: (2024)

Supervised machine learning based signal demodulation in chaotic communications
by: Kozlenko, Mykola
Published: (2025)

Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms
by: Yang, Yahong, et al.
Published: (2025)

Learning from Preferences and Mixed Demonstrations in General Settings
by: Brown, Jason R, et al.
Published: (2025)

A Monad-Based Clause Architecture for Artificial Age Score (AAS) in Large Language Models
by: Kayadibi, Seyma Yaman
Published: (2025)

Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning
by: Stricker, Patrick, et al.
Published: (2026)

Semantic Information G Theory for Range Control with Tradeoff between Purposiveness and Efficiency
by: Lu, Chenguang
Published: (2024)

Spiking Neural Networks for SAR Interferometric Phase Unwrapping: A Theoretical Framework for Energy-Efficient Processing
by: Bara, Marc
Published: (2025)

CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimization
by: Jiang, Yuning, et al.
Published: (2025)

Enhancing Low-Altitude Airspace Security: MLLM-Enabled UAV Intent Recognition
by: Lei, Guangyu, et al.
Published: (2025)

Characterizing Agent-Based Model Dynamics via $ε$-Machines and Kolmogorov-Style Complexity
by: Garrone, Roberto
Published: (2025)

Scalable Cross-Attention Transformer for Cooperative Multi-AP OFDM Uplink Reception
by: Tardy, Xavier, et al.
Published: (2026)

Maximum Entropy Relaxation of Multi-Way Cardinality Constraints for Synthetic Population Generation
by: Pachet, François, et al.
Published: (2026)

CS-SHRED: Enhancing SHRED for Robust Recovery of Spatiotemporal Dynamics
by: da Silva, Romulo B., et al.
Published: (2025)

On the Fundamental Limitations of Decentralized Learnable Reward Shaping in Cooperative Multi-Agent Reinforcement Learning
by: Akella, Aditya
Published: (2025)

Efficient compression of neural networks and datasets
by: Barth, Lukas Silvester, et al.
Published: (2025)

Mastering NIM and Impartial Games with Weak Neural Networks: An AlphaZero-inspired Multi-Frame Approach
by: Riis, Søren
Published: (2024)

From Statistical Disclosure Control to Fair AI: Navigating Fundamental Tradeoffs in Differential Privacy
by: Watson, Adriana
Published: (2026)

VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models
by: Zhang, Haichao, et al.
Published: (2025)

Probabilistic and nonlinear compressive sensing
by: Barth, Lukas Silvester, et al.
Published: (2025)

Differentially Private Feature Release for Wireless Sensing: Adaptive Privacy Budget Allocation on CSI Spectrograms
by: Yilmaz, Ipek Sena, et al.
Published: (2025)

Keys in the Weights: Transformer Authentication Using Model-Bound Latent Representations
by: Okatan, Ayşe S., et al.
Published: (2025)