Copertina

Affinity Is Not Enough: Recovering the Free Energy Principle in Mixture-of-Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Wong, Man Yung
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Neural and Evolutionary Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Mixture of Experts with Soft Nearest Neighbor Loss: Resolving Expert Collapse via Representation Disentanglement
di: Agarap, Abien Fred, et al.
Pubblicazione: (2026)

A Gated Residual Kolmogorov-Arnold Networks for Mixtures of Experts
di: Inzirillo, Hugo, et al.
Pubblicazione: (2024)

Incremental Affinity Propagation based on Cluster Consolidation and Stratification
di: Castano, Silvana, et al.
Pubblicazione: (2024)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
di: Csordás, Róbert, et al.
Pubblicazione: (2023)

MoEUT: Mixture-of-Experts Universal Transformers
di: Csordás, Róbert, et al.
Pubblicazione: (2024)

Stuck-at Faults in ReRAM Neuromorphic Circuit Array and their Correction through Machine Learning
di: Sawal, Vedant, et al.
Pubblicazione: (2024)

Conditional Finite Mixtures of Poisson Distributions for Context-Dependent Neural Correlations
di: Sokoloski, Sacha, et al.
Pubblicazione: (2019)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
di: Zhang, Buze, et al.
Pubblicazione: (2026)

Training a General Spiking Neural Network with Improved Efficiency and Minimum Latency
di: Yao, Yunpeng, et al.
Pubblicazione: (2024)

Hidden Activations Are Not Enough: A General Approach to Neural Network Predictions
di: Leblanc, Samuel, et al.
Pubblicazione: (2024)

Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Experts
di: Kratsios, Anastasis, et al.
Pubblicazione: (2024)

Collapse-Free Prototype Readout Layer for Transformer Encoders
di: Cirrincione, Giansalvo, et al.
Pubblicazione: (2026)

Brain in the Dark: Design Principles for Neuromimetic Inference under the Free Energy Principle
di: Bazargani, Mehran H., et al.
Pubblicazione: (2025)

A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics
di: Li, Josh, et al.
Pubblicazione: (2026)

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
di: Li, Boyan, et al.
Pubblicazione: (2023)

SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network
di: Wang, Kexin, et al.
Pubblicazione: (2024)

GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS
di: Cortês, Gabriel, et al.
Pubblicazione: (2024)

Energy-Efficient Supervised Learning with a Binary Stochastic Forward-Forward Algorithm
di: Jaiswal, Risi, et al.
Pubblicazione: (2025)

Spiking Synaptic Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks
di: Suetake, Kazuma, et al.
Pubblicazione: (2023)

Synaptic Modulation using Interspike Intervals Increases Energy Efficiency of Spiking Neural Networks
di: Adams, Dylan, et al.
Pubblicazione: (2024)

Is In-Context Universality Enough? MLPs are Also Universal In-Context
di: Kratsios, Anastasis, et al.
Pubblicazione: (2025)

SpikingGamma: Surrogate-Gradient Free and Temporally Precise Online Training of Spiking Neural Networks with Smoothed Delays
di: Koopman, Roel, et al.
Pubblicazione: (2026)

Energy-Efficient Eimeria Parasite Detection Using a Two-Stage Spiking Neural Network Architecture
di: García-Vico, Ángel Miguel, et al.
Pubblicazione: (2025)

HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Control
di: Wang, Haoran, et al.
Pubblicazione: (2024)

Robust Energy Consumption Prediction with a Missing Value-Resilient Metaheuristic-based Neural Network in Mobile App Development
di: Mousavirad, Seyed Jalaleddin, et al.
Pubblicazione: (2023)

SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting
di: Tang, Kaiwen, et al.
Pubblicazione: (2026)

Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundary
di: Kumaresan, Ramchand
Pubblicazione: (2026)

EOE: Evolutionary Optimization of Experts for Training Language Models
di: Chen, Yingshi
Pubblicazione: (2025)

Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
di: Tang, Kaiwen, et al.
Pubblicazione: (2024)

Need is All You Need: Homeostatic Neural Networks Adapt to Concept Shift
di: Man, Kingson, et al.
Pubblicazione: (2022)

Deep Recurrent Q-learning for Energy-constrained Coverage with a Mobile Robot
di: Zellner, Aaron, et al.
Pubblicazione: (2022)

Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians
di: Tomihari, Akiyoshi, et al.
Pubblicazione: (2025)

Learning Mixture-of-Experts for General-Purpose Black-Box Discrete Optimization
di: Liu, Shengcai, et al.
Pubblicazione: (2024)

Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study
di: Zhang, Xian-Rong, et al.
Pubblicazione: (2025)

Reconsidering the energy efficiency of spiking neural networks
di: Yan, Zhanglu, et al.
Pubblicazione: (2024)

Differentiable architecture search with multi-dimensional attention for spiking neural networks
di: Man, Yilei, et al.
Pubblicazione: (2024)

Three factor delay learning rules for spiking neural networks
di: Vassallo, Luke, et al.
Pubblicazione: (2026)

Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimization
di: Gao, Zijian, et al.
Pubblicazione: (2026)

Deep Intrinsic Surprise-Regularized Control (DISRC): A Biologically Inspired Mechanism for Efficient Deep Q-Learning in Sparse Environments
di: Kini, Yash, et al.
Pubblicazione: (2026)

SerpentFlow: Generative Unpaired Domain Alignment via Shared-Structure Decomposition
di: Keisler, Julie, et al.
Pubblicazione: (2026)