Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jha, Nandan Kumar, Reagen, Brandon
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Information Theory
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

NerVE: Nonlinear Eigenspectrum Dynamics in LLM Feed-Forward Networks
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2026)

Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2026)

A Random Matrix Theory Perspective on the Learning Dynamics of Multi-head Latent Attention
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2025)

ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2024)

Entropy-Guided Attention for Private LLMs
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2025)

AERO: Entropy-Guided Framework for Private LLM Inference
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2024)

DeepReShape: Redesigning Neural Networks for Efficient Private Inference
di: Jha, Nandan Kumar, et al.
Pubblicazione: (2023)

Graph Community Augmentation with GMM-based Modeling in Latent Space
di: Fukushima, Shintaro, et al.
Pubblicazione: (2024)

Forward-Forward Autoencoder Architectures for Energy-Efficient Wireless Communications
di: Seifert, Daniel, et al.
Pubblicazione: (2025)

A Simple Model of Inference Scaling Laws
di: Levi, Noam
Pubblicazione: (2024)

How Much Is a Dataset Worth? Scaling Laws, the Vendi Score, and Matrix Spectral Functions
di: Bilmes, Jeff A., et al.
Pubblicazione: (2026)

Self-supervised Latent Space Optimization with Nebula Variational Coding
di: Wang, Yida, et al.
Pubblicazione: (2025)

On Observation Time for Recovering Latent Hawkes Networks
di: Linkerhägner, Jonas, et al.
Pubblicazione: (2026)

Latent Space Alignment for Semantic Channel Equalization
di: Hüttebräucker, Tomás, et al.
Pubblicazione: (2024)

LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws
di: Ouyang, Xu, et al.
Pubblicazione: (2026)

Soft Partitioning of Latent Space for Semantic Channel Equalization
di: Hüttebräucker, Tomás, et al.
Pubblicazione: (2024)

Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons
di: Spieler, Aaron, et al.
Pubblicazione: (2026)

Model Merging by Output-Space Projection
di: Evans, Bethan, et al.
Pubblicazione: (2026)

Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models
di: Zhang, Yihan, et al.
Pubblicazione: (2022)

Speculative Decoding Scaling Laws (SDSL): Throughput Optimization Made Simple
di: Bozorgkhoo, Amirhossein, et al.
Pubblicazione: (2026)

Latent Space Characterization of Autoencoder Variants
di: Shrivastava, Anika, et al.
Pubblicazione: (2024)

Learning Capacity: A Measure of the Effective Dimensionality of a Model
di: Chen, Daiwei, et al.
Pubblicazione: (2023)

SCA-LLM: Spectral-Attentive LLM-Based Wireless World Modeling for Agentic Communications
di: He, Ke, et al.
Pubblicazione: (2025)

Latent Space Alignment for AI-Native MIMO Semantic Communications
di: Pandolfo, Mario Edoardo, et al.
Pubblicazione: (2025)

Agentic AI-Based Joint Computing and Networking via Mixture of Experts and Large Language Models
di: Reifert, Robert-Jeron, et al.
Pubblicazione: (2026)

How Big Should a Wireless Foundation Model Be?
di: Cheng, Wei-Lun, et al.
Pubblicazione: (2026)

Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws
di: Pan, Zhixuan, et al.
Pubblicazione: (2025)

TruncFormer: Private LLM Inference Using Only Truncations
di: Yubeaton, Patrick, et al.
Pubblicazione: (2024)

Orthogonal Approximate Message Passing with Optimal Spectral Initializations for Rectangular Spiked Matrix Models
di: Chen, Haohua, et al.
Pubblicazione: (2025)

Learnable Residual-Based Latent Denoising in Semantic Communication
di: Xu, Mingkai, et al.
Pubblicazione: (2025)

Energy-Efficient Federated Edge Learning For Small-Scale Datasets in Large IoT Networks
di: Xie, Haihui, et al.
Pubblicazione: (2026)

DoDo-Code: an Efficient Levenshtein Distance Embedding-based Code for 4-ary IDS Channel
di: Guo, Alan J. X., et al.
Pubblicazione: (2023)

Beyond the Loss Curve: Scaling Laws, Active Learning, and the Limits of Learning from Exact Posteriors
di: Khorasani, Arian, et al.
Pubblicazione: (2026)

Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligence
di: Bennis, Mehdi, et al.
Pubblicazione: (2025)

Rethinking Selectivity in State Space Models: A Minimal Predictive Sufficiency Approach
di: Wang, Yiyi, et al.
Pubblicazione: (2025)

Trace Reconstruction with Language Models
di: Weindel, Franziska, et al.
Pubblicazione: (2025)

Asymptotics of Language Model Alignment
di: Yang, Joy Qiping, et al.
Pubblicazione: (2024)

On the Nonasymptotic Scaling Guarantee of Hyperparameter Estimation in Inhomogeneous, Weakly-Dependent Complex Network Dynamical Systems
di: Yu, Yi, et al.
Pubblicazione: (2026)

Asymptotic Theory of Eigenvectors for Latent Embeddings with Generalized Laplacian Matrices
di: Fan, Jianqing, et al.
Pubblicazione: (2025)

Klein Model for Hyperbolic Neural Networks
di: Mao, Yidan, et al.
Pubblicazione: (2024)