Cover Image

Relational inductive biases on attention mechanisms

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mijangos, Víctor, Gutierrez-Vasques, Ximena, Arriola, Verónica E., Rodríguez-Domínguez, Ulises, Cervantes, Alexis, Almanzara, José Luis
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The in-context inductive biases of vision-language models differ across modalities
by: Allen, Kelsey, et al.
Published: (2025)

Integrating a Heterogeneous Graph with Entity-aware Self-attention using Relative Position Labels for Reading Comprehension Model
by: Foolad, Shima, et al.
Published: (2023)

Modeling citation worthiness by using attention-based bidirectional long short-term memory networks and interpretable models
by: Zeng, Tong, et al.
Published: (2024)

Probing self-attention in self-supervised speech models for cross-linguistic differences
by: Gopinath, Sai, et al.
Published: (2024)

Translution: Unifying Self-attention and Convolution for Adaptive and Relative Modeling
by: Fan, Hehe, et al.
Published: (2025)

Mapping of attention mechanisms to a generalized Potts model
by: Rende, Riccardo, et al.
Published: (2023)

Simple linear attention language models balance the recall-throughput tradeoff
by: Arora, Simran, et al.
Published: (2024)

Insect-inspired modular architectures as inductive biases for reinforcement learning
by: Staples, Anne E.
Published: (2026)

Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers
by: Altabaa, Awni, et al.
Published: (2023)

Can SAEs reveal and mitigate racial biases of LLMs in healthcare?
by: Ahsan, Hiba, et al.
Published: (2025)

Enhancing Sindhi Word Segmentation using Subword Representation Learning and Position-aware Self-attention
by: Ali, Wazir, et al.
Published: (2020)

Protected group bias and stereotypes in Large Language Models
by: Kotek, Hadas, et al.
Published: (2024)

B-score: Detecting biases in large language models using response history
by: Vo, An, et al.
Published: (2025)

HREB-CRF: Hierarchical Reduced-bias EMA for Chinese Named Entity Recognition
by: Sun, Sijin, et al.
Published: (2025)

AtteSTNet -- An attention and subword tokenization based approach for code-switched text hate speech detection
by: Shingi, Geet, et al.
Published: (2021)

A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets
by: Jahin, Md Abrar, et al.
Published: (2024)

TransformerFAM: Feedback attention is working memory
by: Hwang, Dongseong, et al.
Published: (2024)

An investigation of structures responsible for gender bias in BERT and DistilBERT
by: Leteno, Thibaud, et al.
Published: (2024)

Robust Generalization Strategies for Morpheme Glossing in an Endangered Language Documentation Context
by: Ginn, Michael, et al.
Published: (2023)

What are you sinking? A geometric approach on attention sink
by: Ruscio, Valeria, et al.
Published: (2025)

Comparison of different Unique hard attention transformer models by the formal languages they can recognize
by: Ryvkin, Leonid
Published: (2025)

Towards detecting unanticipated bias in Large Language Models
by: Kruspe, Anna
Published: (2024)

A RelEntLess Benchmark for Modelling Graded Relations between Named Entities
by: Ushio, Asahi, et al.
Published: (2023)

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
by: Song, Jiajun, et al.
Published: (2024)

Inducing anxiety in large language models can induce bias
by: Coda-Forno, Julian, et al.
Published: (2023)

On the effectiveness of Large Language Models in the mechanical design domain
by: Grandi, Daniele, et al.
Published: (2025)

Fading memory as inductive bias in residual recurrent networks
by: Dubinin, Igor, et al.
Published: (2023)

ChatGPT for automated grading of short answer questions in mechanical ventilation
by: Jade, Tejas, et al.
Published: (2025)

Speculative Decoding Across Languages
by: Paudel, Nirajan, et al.
Published: (2026)

Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs
by: Gallego, Víctor
Published: (2024)

Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective
by: Gallego, Victor
Published: (2023)

Repetitions are not all alike: distinct mechanisms sustain repetition in language models
by: Mahaut, Matéo, et al.
Published: (2025)

Benchmarking and Understanding Compositional Relational Reasoning of LLMs
by: Ni, Ruikang, et al.
Published: (2024)

Relational Graph Convolutional Networks for Sentiment Analysis
by: Khosravi, Asal, et al.
Published: (2024)

How DDAIR you? Disambiguated Data Augmentation for Intent Recognition
by: Castillo-López, Galo, et al.
Published: (2026)

Interpretation of the Intent Detection Problem as Dynamics in a Low-dimensional Space
by: Sanchez-Karhunen, Eduardo, et al.
Published: (2024)

Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion
by: Rigal, Bruno, et al.
Published: (2026)

TRA: Better Length Generalisation with Threshold Relative Attention
by: Opper, Mattia, et al.
Published: (2025)

Factual Knowledge in Language Models: Robustness and Anomalies under Simple Temporal Context Variations
by: Khodja, Hichem Ammar, et al.
Published: (2025)

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
by: Munkhdalai, Tsendsuren, et al.
Published: (2024)