Cover Image

Brain-Like Language Processing via a Shallow Untrained Multihead Attention Network

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	AlKhamissi, Badr, Tuckute, Greta, Bosselut, Antoine, Schrimpf, Martin
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units
by: AlKhamissi, Badr, et al.
Published: (2024)

From Language to Cognition: How LLMs Outgrow the Human Language Network
by: AlKhamissi, Badr, et al.
Published: (2025)

Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization
by: AlKhamissi, Badr, et al.
Published: (2025)

Instruction-tuning Aligns LLMs to the Human Brain
by: Aw, Khai Loong, et al.
Published: (2023)

Inducing Dyslexia in Vision Language Models
by: Honarmand, Melika, et al.
Published: (2025)

MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding
by: Gokce, Abdulkadir, et al.
Published: (2026)

A Context-Contrastive Inference Approach To Partial Diacritization
by: ElNokrashy, Muhammad, et al.
Published: (2024)

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification
by: ElNokrashy, Muhammad, et al.
Published: (2022)

Rational Metareasoning for Large Language Models
by: De Sabbata, C. Nicolò, et al.
Published: (2024)

Evaluating Contrast Localizer for Identifying Causal Units in Social & Mathematical Tasks in Language Models
by: Jamaa, Yassine, et al.
Published: (2025)

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data
by: AlKhamissi, Badr, et al.
Published: (2024)

"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models
by: Halevy, Karina, et al.
Published: (2024)

Investigating Cultural Alignment of Large Language Models
by: AlKhamissi, Badr, et al.
Published: (2024)

Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
by: AlKhamissi, Mai, et al.
Published: (2025)

Khattat: Enhancing Readability and Concept Representation of Semantic Typography
by: Hussein, Ahmed, et al.
Published: (2024)

TopoLM: brain-like spatio-functional organization in a topographic language model
by: Rathi, Neil, et al.
Published: (2024)

Large Language Models Align with the Human Brain during Creative Thinking
by: Ismayilzada, Mete, et al.
Published: (2026)

Rosetta Stone at KSAA-RD Shared Task: A Hop From Language Modeling To Word--Definition Alignment
by: ElBakry, Ahmed, et al.
Published: (2023)

PERK: Long-Context Reasoning as Parameter-Efficient Test-Time Learning
by: Chen, Zeming, et al.
Published: (2025)

Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining
by: Bayazit, Deniz, et al.
Published: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
by: Foroutan, Negar, et al.
Published: (2025)

ConLID: Supervised Contrastive Learning for Low-Resource Language Identification
by: Foroutan, Negar, et al.
Published: (2025)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
by: Bozic, Vukasin, et al.
Published: (2023)

Model Connectomes: A Generational Approach to Data-Efficient Language Models
by: Kotar, Klemen, et al.
Published: (2025)

Discovering Knowledge-Critical Subnetworks in Pretrained Language Models
by: Bayazit, Deniz, et al.
Published: (2023)

Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness
by: Saha, Shreya, et al.
Published: (2025)

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents
by: Talokar, Nivya, et al.
Published: (2026)

Reliable Evaluation and Benchmarks for Statement Autoformalization
by: Poiroux, Auguste, et al.
Published: (2024)

LLMs Are In-Context Bandit Reinforcement Learners
by: Monea, Giovanni, et al.
Published: (2024)

Evaluating Language Model Agency through Negotiations
by: Davidson, Tim R., et al.
Published: (2024)

Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs
by: Fan, Dongyang, et al.
Published: (2025)

GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters
by: Choudhary, Anand, et al.
Published: (2025)

Introducing HALC: A general pipeline for finding optimal prompting strategies for automated coding with LLMs in the computational social sciences
by: Reich, Andreas, et al.
Published: (2025)

Topoformer: brain-like topographic organization in Transformer language models through spatial querying and reweighting
by: Binhuraib, Taha, et al.
Published: (2025)

A Logical Fallacy-Informed Framework for Argument Generation
by: Mouchel, Luca, et al.
Published: (2024)

MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification
by: Sirbu, Iustin, et al.
Published: (2025)

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization
by: Foroutan, Negar, et al.
Published: (2025)

Large Language and Reasoning Models are Shallow Disjunctive Reasoners
by: Khalid, Irtaza, et al.
Published: (2025)

Why Is RLHF Alignment Shallow? A Gradient Analysis
by: Young, Robin
Published: (2026)

Language models align with brain regions that represent concepts across modalities
by: Ryskina, Maria, et al.
Published: (2025)