Imagen de Portada

NerVE: Nonlinear Eigenspectrum Dynamics in LLM Feed-Forward Networks

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Jha, Nandan Kumar, Reagen, Brandon
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?
por: Jha, Nandan Kumar, et al.
Publicado: (2025)

AERO: Entropy-Guided Framework for Private LLM Inference
por: Jha, Nandan Kumar, et al.
Publicado: (2024)

A Random Matrix Theory Perspective on the Learning Dynamics of Multi-head Latent Attention
por: Jha, Nandan Kumar, et al.
Publicado: (2025)

Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws
por: Jha, Nandan Kumar, et al.
Publicado: (2026)

ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models
por: Jha, Nandan Kumar, et al.
Publicado: (2024)

Entropy-Guided Attention for Private LLMs
por: Jha, Nandan Kumar, et al.
Publicado: (2025)

DeepReShape: Redesigning Neural Networks for Efficient Private Inference
por: Jha, Nandan Kumar, et al.
Publicado: (2023)

TruncFormer: Private LLM Inference Using Only Truncations
por: Yubeaton, Patrick, et al.
Publicado: (2024)

Eigenspectrum Analysis of Neural Networks without Aspect Ratio Bias
por: Hu, Yuanzhe, et al.
Publicado: (2025)

Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
por: Cheng, Tin Sum, et al.
Publicado: (2024)

Quantum Circuit Simulation of Compartmental Drug Dynamics: Leveraging Variational Algorithms for Nonlinear Mixed-Effects Population Pharmacokinetics
por: Singh, Isshaan, et al.
Publicado: (2026)

On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
por: Sun, Haoyuan, et al.
Publicado: (2025)

Network and Compiler Optimizations for Efficient Linear Algebra Kernels in Private Transformer Inference
por: Garimella, Karthik, et al.
Publicado: (2025)

Coherent Feed Forward Quantum Neural Network
por: Singh, Utkarsh, et al.
Publicado: (2024)

Optimizing Dense Feed-Forward Neural Networks
por: Balderas, Luis, et al.
Publicado: (2023)

Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks
por: Omae, Yuto, et al.
Publicado: (2026)

MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for Transformers
por: Jaiswal, Ajay, et al.
Publicado: (2026)

Feed-Forward Latent Domain Adaptation
por: Bohdal, Ondrej, et al.
Publicado: (2022)

Supernodes and Halos: Loss-Critical Hubs in LLM Feed-Forward Layers
por: Cherilyn, Audrey, et al.
Publicado: (2026)

Feed-Forward Optimization With Delayed Feedback for Neural Network Training
por: Flügel, Katharina, et al.
Publicado: (2023)

Flash Multi-Head Feed-Forward Network
por: Zhang, Minshen, et al.
Publicado: (2025)

Feed-Forward Neural Networks as a Mixed-Integer Program
por: Aftabi, Navid, et al.
Publicado: (2024)

A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networks
por: Chickering, Kyle R.
Publicado: (2024)

Merging Feed-Forward Sublayers for Compressed Transformers
por: Verma, Neha, et al.
Publicado: (2025)

FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
por: Jaiswal, Ajay, et al.
Publicado: (2024)

Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Node
por: Charalampopoulos, Andreas, et al.
Publicado: (2024)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
por: Bozic, Vukasin, et al.
Publicado: (2023)

LSTM VS. Feed-Forward Autoencoders for Unsupervised Fault Detection in Hydraulic Pumps
por: Sánchez, P., et al.
Publicado: (2026)

Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers
por: Behrens, Freya, et al.
Publicado: (2024)

Second-Order Forward-Mode Automatic Differentiation for Optimization
por: Cobb, Adam D., et al.
Publicado: (2024)

Parallel Multi-path Feed Forward Neural Networks (PMFFNN) for Long Columnar Datasets: A Novel Approach to Complexity Reduction
por: Jadouli, Ayoub, et al.
Publicado: (2024)

The Discrete Charm of the MLP: Binary Routing of Continuous Signals in Transformer Feed-Forward Layers
por: Balogh, Peter
Publicado: (2026)

CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems
por: Zhang, Haochen, et al.
Publicado: (2025)

FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction
por: Nguyen, Thuan Hoang, et al.
Publicado: (2026)

GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space
por: Attaiki, Souhaib, et al.
Publicado: (2024)

Accelerating Large Language Models through Partially Linear Feed-Forward Network
por: Hu, Gansen, et al.
Publicado: (2025)

VE: Modeling Multivariate Time Series Correlation with Variate Embedding
por: Wang, Shangjiong, et al.
Publicado: (2024)

Protecting Feed-Forward Networks from Adversarial Attacks Using Predictive Coding
por: Ganjidoost, Ehsan, et al.
Publicado: (2024)

FwdLLM: Efficient FedLLM using Forward Gradient
por: Xu, Mengwei, et al.
Publicado: (2023)

BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spaces
por: Landers, Matthew, et al.
Publicado: (2024)