Cover Image

Counting and Algorithmic Generalization with Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ouellette, Simon, Pfister, Rolf, Jud, Hansueli
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI
by: Pfister, Rolf, et al.
Published: (2025)

Classifying bulk-edge anomalies in the Dirac Hamiltonian
by: Jud, Hansueli, et al.
Published: (2024)

Towards Efficient Neurally-Guided Program Induction for ARC-AGI
by: Ouellette, Simon
Published: (2024)

Implicitly Aligning Humans and Autonomous Agents through Shared Task Abstractions
by: Aroca-Ouellette, Stéphane, et al.
Published: (2025)

The Effect of Attention Head Count on Transformer Approximation
by: Yu, Penghao, et al.
Published: (2025)

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
by: Cho, Hanseul, et al.
Published: (2024)

Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical features
by: MacMillan, Theodore, et al.
Published: (2025)

TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
by: Cao, Defu, et al.
Published: (2023)

On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models
by: Horvat, Christian, et al.
Published: (2024)

The Counting Power of Transformers
by: Sälzer, Marco, et al.
Published: (2025)

When Can Transformers Count to n?
by: Yehudai, Gilad, et al.
Published: (2024)

Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers
by: Behrens, Freya, et al.
Published: (2024)

Learning Randomized Algorithms with Transformers
by: von Oswald, Johannes, et al.
Published: (2024)

Invariance-Based Dynamic Regret Minimization
by: Lazzaretto, Margherita, et al.
Published: (2026)

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
by: Dao, Tri, et al.
Published: (2024)

Aligning LLMs by Predicting Preferences from User Writing Samples
by: Aroca-Ouellette, Stéphane, et al.
Published: (2025)

Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers
by: Yang, Andy, et al.
Published: (2024)

Extrapolation-Aware Nonparametric Statistical Inference
by: Pfister, Niklas, et al.
Published: (2024)

Identifiability of Sparse Causal Effects using Instrumental Variables
by: Pfister, Niklas, et al.
Published: (2022)

Information-Theoretic Proofs for Diffusion Sampling
by: Reeves, Galen, et al.
Published: (2025)

Neural Polar Decoders for DNA Data Storage
by: Aharoni, Ziv, et al.
Published: (2025)

Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task
by: Golkar, Siavash, et al.
Published: (2024)

Clustering of Indonesian and Western Gamelan Orchestras through Machine Learning of Performance Parameters
by: Linke, Simon, et al.
Published: (2024)

CountsDiff: A Diffusion Model on the Natural Numbers for Generation and Imputation of Count-Based Data
by: Soatto, Renzo G., et al.
Published: (2026)

How to Count Coughs: An Event-Based Framework for Evaluating Automatic Cough Detection Algorithm Performance
by: Orlandic, Lara, et al.
Published: (2024)

Boosted Control Functions: Distribution generalization and invariance in confounded models
by: Gnecco, Nicola, et al.
Published: (2023)

Learned Feature Importance Scores for Automated Feature Engineering
by: Dong, Yihe, et al.
Published: (2024)

Perturbation-based Effect Measures for Compositional Data
by: Lundborg, Anton Rask, et al.
Published: (2023)

Transformers versus the EM Algorithm in Multi-class Clustering
by: He, Yihan, et al.
Published: (2025)

Spectral Conditioning of Attention Improves Transformer Performance
by: Saratchandran, Hemanth, et al.
Published: (2026)

Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers
by: Chakrabarti, Sayak, et al.
Published: (2026)

Search-Adaptor: Embedding Customization for Information Retrieval
by: Yoon, Jinsung, et al.
Published: (2023)

Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning
by: Han, Sungwon, et al.
Published: (2024)

CountARFactuals -- Generating plausible model-agnostic counterfactual explanations with adversarial random forests
by: Dandl, Susanne, et al.
Published: (2024)

Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixtures
by: Chen, Zhiheng, et al.
Published: (2025)

Finding Clustering Algorithms in the Transformer Architecture
by: Clarkson, Kenneth L., et al.
Published: (2025)

Transformers meet Neural Algorithmic Reasoners
by: Bounsi, Wilfried, et al.
Published: (2024)

Neural Polar Decoders for Deletion Channels
by: Aharoni, Ziv, et al.
Published: (2025)

Identifying Causal Effects Using a Single Proxy Variable
by: Vollmer, Silvan, et al.
Published: (2026)

Invariant Subspace Decomposition
by: Lazzaretto, Margherita, et al.
Published: (2024)