Image de couverture de livre

A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lubana, Ekdeep Singh, Kawaguchi, Kyogo, Dick, Robert P., Tanaka, Hidenori
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space
par: Park, Core Francisco, et autres
Publié: (2024)

Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model
par: Khona, Mikail, et autres
Publié: (2024)

Emergence of Hierarchical Emotion Organization in Large Language Models
par: Zhao, Bo, et autres
Publié: (2025)

In-Context Learning Dynamics with Random Binary Sequences
par: Bigelow, Eric J., et autres
Publié: (2023)

Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
par: Okawa, Maya, et autres
Publié: (2023)

Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
par: Ramesh, Rahul, et autres
Publié: (2023)

In-Context Learning Strategies Emerge Rationally
par: Wurgaft, Daniel, et autres
Publié: (2025)

Analyzing (In)Abilities of SAEs via Formal Languages
par: Menon, Abhinav, et autres
Publié: (2024)

How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations
par: Jaipersaud, Brandon, et autres
Publié: (2025)

Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
par: Bigelow, Eric, et autres
Publié: (2025)

The Impact of Off-Policy Training Data on Probe Generalisation
par: Kirch, Nathalie, et autres
Publié: (2025)

ICLR: In-Context Learning of Representations
par: Park, Core Francisco, et autres
Publié: (2024)

Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry
par: Hindupur, Sai Sumedh R., et autres
Publié: (2025)

Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing
par: Nishi, Kento, et autres
Publié: (2024)

Competition Dynamics Shape Algorithmic Phases of In-Context Learning
par: Park, Core Francisco, et autres
Publié: (2024)

From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?
par: Mueller, Aaron, et autres
Publié: (2025)

Abrupt Learning in Transformers: A Case Study on Matrix Completion
par: Gopalani, Pulkit, et autres
Publié: (2024)

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks
par: Jain, Samyak, et autres
Publié: (2023)

Continuous-Time Analysis of Adaptive Optimization and Normalization
par: Gould, Rhys, et autres
Publié: (2024)

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
par: Bigelow, Eric, et autres
Publié: (2026)

Swing-by Dynamics in Concept Learning and Compositional Generalization
par: Yang, Yongyi, et autres
Publié: (2024)

Do Sparse Autoencoders Capture Concept Manifolds?
par: Bhalla, Usha, et autres
Publié: (2026)

Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework
par: Xia, Yuan, et autres
Publié: (2025)

FormalMATH: Benchmarking Formal Mathematical Reasoning of Large Language Models
par: Yu, Zhouliang, et autres
Publié: (2025)

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
par: Mistry, Deven Mahesh, et autres
Publié: (2025)

Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders
par: Bohacek, Matyas, et autres
Publié: (2025)

Towards Reliable Evaluation of Behavior Steering Interventions in LLMs
par: Pres, Itamar, et autres
Publié: (2024)

Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics
par: Zur, Amir, et autres
Publié: (2025)

$\textit{New News}$: System-2 Fine-tuning for Robust Integration of New Knowledge
par: Park, Core Francisco, et autres
Publié: (2025)

Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models
par: Shi, Yubin, et autres
Publié: (2024)

Analyzing Generalization in Pre-Trained Symbolic Regression
par: Voigt, Henrik, et autres
Publié: (2025)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
par: Yao, Jinghan, et autres
Publié: (2024)

Analyzing Memorization in Large Language Models through the Lens of Model Attribution
par: Menta, Tarun Ram, et autres
Publié: (2025)

Forking Paths in Neural Text Generation
par: Bigelow, Eric, et autres
Publié: (2024)

Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models
par: Ding, Zeyang, et autres
Publié: (2026)

Towards a Formal Creativity Theory: Preliminary results in Novelty and Transformativeness
par: Santo, Luís Espírito, et autres
Publié: (2024)

To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers
par: Xu, Kevin, et autres
Publié: (2025)

Marabou 2.0: A Versatile Formal Analyzer of Neural Networks
par: Wu, Haoze, et autres
Publié: (2024)

Emergency Preemption Without Online Exploration: A Decision Transformer Approach
par: Su, Haoran, et autres
Publié: (2026)

Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards
par: Padula, Alexander G., et autres
Publié: (2024)