Cover Image

Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Altabaa, Awni, Webb, Taylor, Cohen, Jonathan, Lafferty, John
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Approximation of relation functions and attention mechanisms
by: Altabaa, Awni, et al.
Published: (2024)

Disentangling and Integrating Relational and Sensory Information in Transformer Architectures
by: Altabaa, Awni, et al.
Published: (2024)

Learning Hierarchical Relational Representations through Relational Convolutions
by: Altabaa, Awni, et al.
Published: (2023)

CoT Information: Improved Sample Complexity under Chain-of-Thought Supervision
by: Altabaa, Awni, et al.
Published: (2025)

Unlocking Out-of-Distribution Generalization in Transformers via Recursive Latent Space Reasoning
by: Altabaa, Awni, et al.
Published: (2025)

Slot Abstractors: Toward Scalable Abstract Visual Reasoning
by: Mondal, Shanka Subhra, et al.
Published: (2024)

On the Role of Information Structure in Reinforcement Learning for Partially-Observable Sequential Teams and Games
by: Altabaa, Awni, et al.
Published: (2024)

Decentralized Multi-Agent Reinforcement Learning for Continuous-Space Stochastic Games
by: Altabaa, Awni, et al.
Published: (2023)

Relational inductive biases on attention mechanisms
by: Mijangos, Víctor, et al.
Published: (2025)

Relational reasoning and inductive bias in transformers and large language models
by: Geerts, Jesse, et al.
Published: (2025)

Determinantal Point Process Attention Over Grid Cell Code Supports Out of Distribution Generalization
by: Mondal, Shanka Subhra, et al.
Published: (2023)

Insect-inspired modular architectures as inductive biases for reinforcement learning
by: Staples, Anne E.
Published: (2026)

On the inductive bias of infinite-depth ResNets and the bottleneck rank
by: Boix-Adsera, Enric
Published: (2025)

Generalization through variance: how noise shapes inductive biases in diffusion models
by: Vastola, John J.
Published: (2025)

Fading memory as inductive bias in residual recurrent networks
by: Dubinin, Igor, et al.
Published: (2023)

Abstractors Manual. Fourth Edition.
by: Oldsen, Carl F.
Published: (1970)

Chemist-aligned retrosynthesis by ensembling diverse inductive bias models
by: Maziarz, Krzysztof, et al.
Published: (2024)

The Relational Bottleneck as an Inductive Bias for Efficient Abstraction
by: Webb, Taylor W., et al.
Published: (2023)

A standard transformer and attention with linear biases for molecular conformer generation
by: Gurev, Viatcheslav, et al.
Published: (2025)

Spatially-informed transformers: Injecting geostatistical covariance biases into self-attention for spatio-temporal forecasting
by: Calleo, Yuri
Published: (2025)

Anomaly detection in time-series via inductive biases in the latent space of conditional normalizing flows
by: Baumgartner, David, et al.
Published: (2026)

Need a Small Specialized Language Model? Plan Early!
by: Grangier, David, et al.
Published: (2024)

Compute-Optimal Quantization-Aware Training
by: Dremov, Aleksandr, et al.
Published: (2025)

Probabilistic unifying relations for modelling epistemic and aleatoric uncertainty: semantics and automated reasoning with theorem proving
by: Ye, Kangfeng, et al.
Published: (2023)

The in-context inductive biases of vision-language models differ across modalities
by: Allen, Kelsey, et al.
Published: (2025)

On the expressiveness and spectral bias of KANs
by: Wang, Yixuan, et al.
Published: (2024)

Graph neural networks with configuration cross-attention for tensor compilers
by: Khizbullin, Dmitrii, et al.
Published: (2024)

Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
by: Lee, Ivan, et al.
Published: (2023)

Making medical vision-language models think causally across modalities with retrieval-augmented cross-modal reasoning
by: Yang, Weiqin, et al.
Published: (2026)

A result relating convex n-widths to covering numbers with some applications to neural networks
by: Baxter, Jonathan, et al.
Published: (2025)

S-JEPA: towards seamless cross-dataset transfer through dynamic spatial attention
by: Guetschel, Pierre, et al.
Published: (2024)

Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning
by: He, Pengfei, et al.
Published: (2024)

Self-attentive Transformer for Fast and Accurate Postprocessing of Temperature and Wind Speed Forecasts
by: Van Poecke, Aaron, et al.
Published: (2024)

Distributional bias compromises leave-one-out cross-validation
by: Austin, George I., et al.
Published: (2024)

The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training
by: Saponati, Matteo, et al.
Published: (2025)

Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
by: McGreivy, Nick, et al.
Published: (2024)

TransformerFAM: Feedback attention is working memory
by: Hwang, Dongseong, et al.
Published: (2024)

Composing diffusion priors with explicit physical context via generative Gibbs sampling
by: Wang, Weizhou, et al.
Published: (2026)

A unified cross-attention model for predicting antigen binding specificity to both HLA and TCR molecules
by: Yu, Chenpeng, et al.
Published: (2024)

Easy attention: A simple attention mechanism for temporal predictions with transformers
by: Sanchis-Agudo, Marcial, et al.
Published: (2023)