Cover Image

When can transformers reason with abstract symbols?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Boix-Adsera, Enric, Saremi, Omid, Abbe, Emmanuel, Bengio, Samy, Littwin, Etai, Susskind, Joshua
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Vanishing Gradients in Reinforcement Finetuning of Language Models
by: Razin, Noam, et al.
Published: (2023)

How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad
by: Abbe, Emmanuel, et al.
Published: (2024)

Secret mixtures of experts inside your LLM
by: Boix-Adsera, Enric
Published: (2025)

On the inductive bias of infinite-depth ResNets and the bottleneck rank
by: Boix-Adsera, Enric
Published: (2025)

Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones
by: Mirtaheri, Parsa, et al.
Published: (2025)

Towards a theory of model distillation
by: Boix-Adsera, Enric
Published: (2024)

The power of fine-grained experts: Granularity boosts expressivity in Mixture of Experts
by: Boix-Adsera, Enric, et al.
Published: (2025)

Toward universal steering and monitoring of AI models
by: Beaglehole, Daniel, et al.
Published: (2025)

To Infinity and Beyond: Tool-Use Unlocks Length Generalization in State Space Models
by: Malach, Eran, et al.
Published: (2025)

The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks
by: Abbe, Emmanuel, et al.
Published: (2022)

How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks
by: Littwin, Etai, et al.
Published: (2024)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity
by: Fu, Yicheng, et al.
Published: (2024)

The Features at Convergence Theorem: a first-principles alternative to the Neural Feature Ansatz for how networks learn representations
by: Boix-Adsera, Enric, et al.
Published: (2025)

AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking
by: Gao, Silin, et al.
Published: (2025)

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
by: Sprague, Zayne, et al.
Published: (2024)

RL for Reasoning by Adaptively Revealing Rationales
by: Amani, Mohammad Hossein, et al.
Published: (2025)

Neural networks for abstraction and reasoning: Towards broad generalization in machines
by: Bober-Irizar, Mikel, et al.
Published: (2024)

The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
by: Shojaee, Parshin, et al.
Published: (2025)

Action abstractions for amortized sampling
by: Boussif, Oussama, et al.
Published: (2024)

LLMs cannot find reasoning errors, but can correct them given the error location
by: Tyen, Gladys, et al.
Published: (2023)

Reasoning's Razor: Reasoning Improves Accuracy but Can Hurt Recall at Critical Operating Points in Safety and Hallucination Detection
by: Chegini, Atoosa, et al.
Published: (2025)

Generalization on the Unseen, Logic Reasoning and Degree Curriculum
by: Abbe, Emmanuel, et al.
Published: (2023)

Boolformer: Symbolic Regression of Logic Functions with Transformers
by: d'Ascoli, Stéphane, et al.
Published: (2023)

When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks
by: Lo, Chung-Hsiang, et al.
Published: (2026)

Text-Conditional JEPA for Learning Semantically Rich Visual Representations
by: Huang, Chen, et al.
Published: (2026)

Goldilocks RL: Tuning Task Difficulty to Escape Sparse Rewards for Reasoning
by: Mahrooghi, Ilia, et al.
Published: (2026)

Discrete, compositional, and symbolic representations through attractor dynamics
by: Nam, Andrew, et al.
Published: (2023)

A Complexity-Based Theory of Compositionality
by: Elmoznino, Eric, et al.
Published: (2024)

Artificial Expert Intelligence through PAC-reasoning
by: Shalev-Shwartz, Shai, et al.
Published: (2024)

What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?
by: Zhang, Yizhe, et al.
Published: (2025)

$k$-server-bench: Automating Potential Discovery for the $k$-Server Conjecture
by: Brilliantov, Kirill, et al.
Published: (2026)

Is continuous CoT better suited for multi-lingual reasoning?
by: Bashir, Ali Hamza, et al.
Published: (2026)

Sudoku-Bench: Evaluating creative reasoning with Sudoku variants
by: Seely, Jeffrey, et al.
Published: (2025)

Are complicated loss functions necessary for teaching LLMs to reason?
by: Carrino, Gabriele, et al.
Published: (2026)

Neural Network Parameter-optimization of Gaussian pmDAGs
by: Saremi, Mehrzad
Published: (2023)

Learning logic programs by discovering higher-order abstractions
by: Hocquette, Céline, et al.
Published: (2023)

Your thoughts tell who you are: Characterize the reasoning patterns of LRMs
by: Chen, Yida, et al.
Published: (2025)

Language models show human-like content effects on reasoning tasks
by: Dasgupta, Ishita, et al.
Published: (2022)

Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning
by: Littwin, Etai, et al.
Published: (2024)