Cover Image

Attention to Order: Transformers Discover Phase Transitions via Learnability

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Özönder, Şener
Format:	Preprint
Published:	2025
Subjects:	Materials Science Disordered Systems and Neural Networks Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rapid Discovery of Graphene Nanocrystals Using DFT and Bayesian Optimization with Neural Network Kernel
by: Özönder, Şener, et al.
Published: (2022)

Graph Learning Metallic Glass Discovery from Wikipedia
by: Ouyang, K. -C., et al.
Published: (2025)

SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning
by: Ghafarollahi, Alireza, et al.
Published: (2024)

Autonomous Inorganic Materials Discovery via Multi-Agent Physics-Aware Scientific Reasoning
by: Ghafarollahi, Alireza, et al.
Published: (2025)

Grokking as a First Order Phase Transition in Two Layer Networks
by: Rubin, Noa, et al.
Published: (2023)

How Do Transformers "Do" Physics? Investigating the Simple Harmonic Oscillator
by: Kantamneni, Subhash, et al.
Published: (2024)

Discover physical concepts and equations with machine learning
by: Li, Bao-Bing, et al.
Published: (2024)

Grokking as Dimensional Phase Transition in Neural Networks
by: Wang, Ping
Published: (2026)

Preisach Attention: A Hysteretic Model of Sequential Memory
by: Frydrych, Piotr
Published: (2026)

Field-Induced Ferroelectric Phase Transition Dynamics in PMN-PT compositions near the Morphotropic Phase Boundary
by: Chanan, Shivjeet, et al.
Published: (2026)

Discovering new photovoltaics using optimal transport theory
by: Walker, Matthew A. H., et al.
Published: (2026)

STEM Diffraction Pattern Analysis with Deep Learning Networks
by: Wissel, Sebastian, et al.
Published: (2025)

Deep Generative Learning of Magnetic Frustration in Artificial Spin Ice from Magnetic Force Microscopy Images
by: Neogi, Arnab, et al.
Published: (2025)

Bayesian Optimization of Multi-Bit Pulse Encoding in In2O3/Al2O3 Thin-film Transistors for Temporal Data Processing
by: Meza-Arroyo, Javier, et al.
Published: (2025)

Interpretation of Crystal Energy Landscapes with Kolmogorov-Arnold Networks
by: Zu, Gen, et al.
Published: (2026)

Predicting and Interpreting Energy Barriers of Metallic Glasses with Graph Neural Networks
by: Li, Haoyu, et al.
Published: (2023)

Improving Electrolyte Performance for Target Cathode Loading Using Interpretable Data-Driven Approach
by: Sharma, Vidushi, et al.
Published: (2024)

Generative Inversion of Spectroscopic Data for Amorphous Structure Elucidation
by: Guo, Jiawei, et al.
Published: (2026)

Molecular Identification from AFM images using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks
by: Carracedo-Cosme, Jaime, et al.
Published: (2022)

Data-Driven Learnability Transition of Measurement-Induced Entanglement
by: Qian, Dongheng, et al.
Published: (2025)

Critical Phase Transition in Large Language Models
by: Nakaishi, Kai, et al.
Published: (2024)

Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
by: Defilippis, Leonardo, et al.
Published: (2025)

Generalization through variance: how noise shapes inductive biases in diffusion models
by: Vastola, John J.
Published: (2025)

Identifying internal patterns in (1+1)-dimensional directed percolation using neural networks
by: Parkhomenko, Danil, et al.
Published: (2025)

Grokking vs. Learning: Same Features, Different Encodings
by: Manning-Coe, Dmitry, et al.
Published: (2025)

A Spin Glass Characterization of Neural Networks
by: Li, Jun
Published: (2025)

Applications of Statistical Field Theory in Deep Learning
by: Ringel, Zohar, et al.
Published: (2025)

A Geometric Perspective on the Difficulties of Learning GNN-based SAT Solvers
by: Skenderi, Geri
Published: (2025)

Representation Learning on a Random Lattice
by: Brill, Aryeh
Published: (2025)

Towards Distributed Neural Architectures
by: Cowsik, Aditya, et al.
Published: (2025)

Parameter Symmetry Potentially Unifies Deep Learning Theory
by: Ziyin, Liu, et al.
Published: (2025)

A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs
by: Kalra, Dayal Singh, et al.
Published: (2026)

Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate
by: Kalra, Dayal Singh, et al.
Published: (2026)

A method for quantifying the generalization capabilities of generative models for solving Ising models
by: Ma, Qunlong, et al.
Published: (2024)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
by: Lauditi, Clarissa, et al.
Published: (2026)

KAN: Kolmogorov-Arnold Networks
by: Liu, Ziming, et al.
Published: (2024)

On the origin of neural scaling laws: from random graphs to natural language
by: Barkeshli, Maissam, et al.
Published: (2026)

The Persian Rug: solving toy models of superposition using large-scale symmetries
by: Cowsik, Aditya, et al.
Published: (2024)

Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Model
by: Levi, Noam
Published: (2026)

More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)
by: Meir, Sagi, et al.
Published: (2026)