Imagen de Portada

Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Niu, Xueyan, Bai, Bo, Deng, Lei, Han, Wei
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training
por: Niu, Xueyan, et al.
Publicado: (2026)

High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models
por: Yilmaz, Selim F., et al.
Publicado: (2023)

Muon in Associative Memory Learning: Training Dynamics and Scaling Laws
por: Li, Binghui, et al.
Publicado: (2026)

Understanding Transformer from the Perspective of Associative Memory
por: Zhong, Shu, et al.
Publicado: (2025)

Scaling Laws for Associative Memories
por: Cabannes, Vivien, et al.
Publicado: (2023)

Understanding Factual Recall in Transformers via Associative Memories
por: Nichani, Eshaan, et al.
Publicado: (2024)

Tensor Cache: Eviction-conditioned Associative Memory for Transformers
por: Swain, Kabir, et al.
Publicado: (2026)

Configuration-to-Performance Scaling Law with Neural Ansatz
por: Zhang, Huaqing, et al.
Publicado: (2026)

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
por: Hägele, Alexander, et al.
Publicado: (2024)

Scaling Laws for Data-Efficient Visual Transfer Learning
por: Yang, Wenxuan, et al.
Publicado: (2025)

On the Invariance and Generality of Neural Scaling Laws
por: Han, Xing, et al.
Publicado: (2026)

Determinant Estimation under Memory Constraints and Neural Scaling Laws
por: Ameli, Siavash, et al.
Publicado: (2025)

Understanding and Mitigating the Bias in Sample Selection for Learning with Noisy Labels
por: Wei, Qi, et al.
Publicado: (2024)

Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics
por: Kendiukhov, Ihor
Publicado: (2026)

Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory
por: Kim, Juno, et al.
Publicado: (2026)

Scaling Law for Time Series Forecasting
por: Shi, Jingzhe, et al.
Publicado: (2024)

A Mean Field Ansatz for Zero-Shot Weight Transfer
por: Chen, Xingyuan, et al.
Publicado: (2024)

Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
por: Havrilla, Alex, et al.
Publicado: (2024)

xLSTM Scaling Laws: Competitive Performance with Linear Time-Complexity
por: Beck, Maximilian, et al.
Publicado: (2025)

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
por: Sardana, Nikhil, et al.
Publicado: (2023)

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
por: Wu, Xiaojun, et al.
Publicado: (2025)

From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction
por: Yan, Bencheng, et al.
Publicado: (2025)

Zero-Shot Performance Prediction for Probabilistic Scaling Laws
por: Schram, Viktoria, et al.
Publicado: (2025)

Generalization and Scaling Laws for Mixture-of-Experts Transformers
por: Mayaki, Mansour Zoubeirou a
Publicado: (2026)

Variational Linear Attention: Stable Associative Memory for Long-Context Transformers
por: Pandey, Vishal, et al.
Publicado: (2026)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
por: Liu, Lei, et al.
Publicado: (2025)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
por: Qiu, Zeju, et al.
Publicado: (2026)

GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
por: Liu, Jiaxu, et al.
Publicado: (2025)

Holistic Scaling Laws for Optimal Mixture-of-Experts Architecture Optimization
por: Wan, Weilin, et al.
Publicado: (2026)

Joint Memory Frequency and Computing Frequency Scaling for Energy-efficient DNN Inference
por: Han, Yunchu, et al.
Publicado: (2025)

Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World
por: Bryant, Christopher M., et al.
Publicado: (2026)

Scaling Laws for Predicting Downstream Performance in LLMs
por: Chen, Yangyi, et al.
Publicado: (2024)

Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories
por: Alessandrelli, Andrea, et al.
Publicado: (2025)

Empowering LLMs in Decision Games through Algorithmic Data Synthesis
por: Wang, Haolin, et al.
Publicado: (2025)

From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme
por: Li, Xueyan, et al.
Publicado: (2025)

Scaling Laws for Downstream Task Performance of Large Language Models
por: Isik, Berivan, et al.
Publicado: (2024)

Transformers as Measure-Theoretic Associative Memory: A Statistical Perspective and Minimax Optimality
por: Kawata, Ryotaro, et al.
Publicado: (2026)

Interdomain Attention: Beyond Token-Level Key-Value Memory
por: Kiyohara, Naoki, et al.
Publicado: (2026)

Scaling Laws are Redundancy Laws
por: Bi, Yuda, et al.
Publicado: (2025)

Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes
por: Hu, Jerry Yao-Chieh, et al.
Publicado: (2024)