Cover Image

Superposition Yields Robust Neural Scaling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yizhou, Liu, Ziming, Gore, Jeff
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Universal One-third Time Scaling in Learning Peaked Distributions
by: Liu, Yizhou, et al.
Published: (2026)

A Resource Model For Neural Scaling Law
by: Song, Jinyeop, et al.
Published: (2024)

Inverse Depth Scaling From Most Layers Being Similar
by: Liu, Yizhou, et al.
Published: (2026)

FOCUS: First Order Concentrated Updating Scheme
by: Liu, Yizhou, et al.
Published: (2025)

Neural Thermodynamic Laws for Large Language Model Training
by: Liu, Ziming, et al.
Published: (2025)

Superposition unifies power-law training dynamics
by: Chen, Zixin Jessie, et al.
Published: (2026)

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
by: Han, Seungwook, et al.
Published: (2024)

Physics of Skill Learning
by: Liu, Ziming, et al.
Published: (2025)

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
by: Merth, Thomas, et al.
Published: (2024)

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
by: Xiong, Zheyang, et al.
Published: (2024)

A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
by: Javidnia, Hossein
Published: (2026)

Relative-Based Scaling Law for Neural Language Models
by: Yue, Baoqing, et al.
Published: (2025)

An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Models
by: Zhang, Yizhou, et al.
Published: (2024)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

What Scales in Cross-Entropy Scaling Law?
by: Yan, Junxi, et al.
Published: (2025)

ALICE: An Interpretable Neural Architecture for Generalization in Substitution Ciphers
by: Shen, Jeff, et al.
Published: (2025)

CaRT: Teaching LLM Agents to Know When They Know Enough
by: Liu, Grace, et al.
Published: (2025)

Aggregated Knowledge Model: Enhancing Domain-Specific QA with Fine-Tuned and Retrieval-Augmented Generation Models
by: Liu, Fengchen, et al.
Published: (2024)

Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling
by: Tamoyan, Hovhannes, et al.
Published: (2025)

LIMR: Less is More for RL Scaling
by: Li, Xuefeng, et al.
Published: (2025)

Distantly-Supervised Joint Extraction with Noise-Robust Learning
by: Li, Yufei, et al.
Published: (2023)

SPINAL -- Scaling-law and Preference Integration in Neural Alignment Layers
by: Das, Arion, et al.
Published: (2026)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
by: Qiu, Zeju, et al.
Published: (2026)

Parallel Structures in Pre-training Data Yield In-Context Learning
by: Chen, Yanda, et al.
Published: (2024)

Interpretable by AI Mother Tongue: Native Symbolic Reasoning in Neural Models
by: Liu, Hung Ming
Published: (2025)

Inference-Time Scaling for Generalist Reward Modeling
by: Liu, Zijun, et al.
Published: (2025)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
by: Zhou, Fan, et al.
Published: (2024)

Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
by: Shi, Chengshuai, et al.
Published: (2026)

MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
by: Liu, Qin, et al.
Published: (2025)

Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective
by: Liu, Yu-An, et al.
Published: (2024)

A Single Linear Layer Yields Task-Adapted Low-Rank Matrices
by: Kim, Hwichan, et al.
Published: (2024)

OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
by: Wang, Zengzhi, et al.
Published: (2025)

Automated Capability Discovery via Foundation Model Self-Exploration
by: Lu, Cong, et al.
Published: (2025)

Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models
by: Lu, Cong, et al.
Published: (2024)

Efficient Test-Time Scaling via Self-Calibration
by: Huang, Chengsong, et al.
Published: (2025)

Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
by: Cheng, Ruoxi, et al.
Published: (2025)

MathPile: A Billion-Token-Scale Pretraining Corpus for Math
by: Wang, Zengzhi, et al.
Published: (2023)

Batched Contextual Reinforcement: A Task-Scaling Law for Efficient Reasoning
by: Yang, Bangji, et al.
Published: (2026)

Exploring Scaling Laws for EHR Foundation Models
by: Zhang, Sheng, et al.
Published: (2025)

Scaling Embeddings Outperforms Scaling Experts in Language Models
by: Liu, Hong, et al.
Published: (2026)