Cover Image

From Kepler to Newton: Inductive Biases Guide Learned World Models in Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Ziming, Sanborn, Sophia, Ganguli, Surya, Tolias, Andreas
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Classical Physics
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Physics of Learning: From Autoencoders to Truly Autonomous Learning Machines
by: Ushveridze, Alex
Published: (2024)

A hybrid numerical methodology coupling Reduced Order Modeling and Graph Neural Networks for non-parametric geometries: applications to structural dynamics problems
by: Matray, Victor, et al.
Published: (2024)

AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge
by: Fang, You-Le, et al.
Published: (2025)

Is Phase Really Needed for Weakly-Supervised Dereverberation ?
by: Rodrigues, Marius, et al.
Published: (2025)

Instilling Inductive Biases with Subnetworks
by: Zhang, Enyan, et al.
Published: (2023)

A Physical Theory of Backpropagation: Exact Gradients from the Least-Action Principle
by: Scurria, Antonino Emanuele
Published: (2026)

From Molecules to Mixtures: Learning Representations of Olfactory Mixture Similarity using Inductive Biases
by: Tom, Gary, et al.
Published: (2025)

A General Framework for Robust G-Invariance in G-Equivariant Networks
by: Sanborn, Sophia, et al.
Published: (2023)

Equilibrium Propagation for Non-Conservative Systems
by: Scurria, Antonino Emanuele, et al.
Published: (2026)

Deriving Neural Scaling Laws from the statistics of natural language
by: Cagnetta, Francesco, et al.
Published: (2026)

Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks
by: Chen, Feng, et al.
Published: (2023)

Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks
by: Marchetti, Giovanni Luca, et al.
Published: (2023)

SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases
by: Liu, Yang, et al.
Published: (2023)

Clustering Inductive Biases with Unrolled Networks
by: Huml, Jonathan, et al.
Published: (2023)

On the Inductive Biases of Demographic Parity-based Fair Learning Algorithms
by: Lei, Haoyu, et al.
Published: (2024)

Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning
by: Chen, Feng, et al.
Published: (2025)

Critical attention scaling in long-context transformers
by: Chen, Shi, et al.
Published: (2025)

Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute
by: Roye-Azar, Antonio, et al.
Published: (2025)

Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning
by: Dijujin, Negin Hashemi, et al.
Published: (2025)

TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
by: Ganguli, Anurup
Published: (2026)

An analytic theory of creativity in convolutional diffusion models
by: Kamb, Mason, et al.
Published: (2024)

Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models
by: Morales-Pastor, Adrián, et al.
Published: (2024)

Contrastive Concept-Tree Search for LLM-Assisted Algorithm Discovery
by: Leleu, Timothee, et al.
Published: (2026)

Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
by: Li, Pingzhi, et al.
Published: (2024)

Constants of motion network revisited
by: Fang, Wenqi, et al.
Published: (2025)

Hamiltonian-based neural networks for systems under nonholonomic constraints
by: T., Ignacio Puiggros, et al.
Published: (2024)

Continual Deep Learning on the Edge via Stochastic Local Competition among Subnetworks
by: Christophides, Theodoros, et al.
Published: (2024)

Interpretable Machine Learning in Physics: A Review
by: Wetzel, Sebastian Johann, et al.
Published: (2025)

Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
by: Kuciński, Łukasz, et al.
Published: (2021)

EMVD dataset: a dataset of extreme vocal distortion techniques used in heavy metal
by: Tailleur, Modan, et al.
Published: (2024)

NeuroAI for AI Safety
by: Mineault, Patrick, et al.
Published: (2024)

Leveraging Geometric Visual Illusions as Perceptual Inductive Biases for Vision Models
by: Yang, Haobo, et al.
Published: (2025)

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
by: Vafa, Keyon, et al.
Published: (2025)

Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models
by: Ren, Zirui, et al.
Published: (2026)

Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning
by: Kunin, Daniel, et al.
Published: (2024)

Small Models, Strong Priors: Architectural Inductive Bias for Parameter-Efficient Neural PDE Solvers
by: Sankaran, Shyam, et al.
Published: (2026)

SE(3)-bi-equivariant Transformers for Point Cloud Assembly
by: Wang, Ziming, et al.
Published: (2024)

About the Keplerization of motion in any central force field
by: Carimalo, Christian
Published: (2025)

Linearization of Newton's second law
by: Paliathanasis, Andronikos
Published: (2024)

Universality of physical neural networks with multivariate nonlinearity
by: Savinson, Benjamin, et al.
Published: (2025)