Cover Image

Language Models Implement Simple Word2Vec-style Vector Arithmetic

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Merullo, Jack, Eickhoff, Carsten, Pavlick, Ellie
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Circuit Component Reuse Across Tasks in Transformer Language Models
by: Merullo, Jack, et al.
Published: (2023)

Talking Heads: Understanding Inter-layer Communication in Transformer Language Models
by: Merullo, Jack, et al.
Published: (2024)

Transferring Linear Features Across Language Models With Model Stitching
by: Chen, Alan, et al.
Published: (2025)

Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting
by: Anand, Suraj, et al.
Published: (2024)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
by: Khandelwal, Apoorv, et al.
Published: (2024)

Does Training on Synthetic Data Make Models Less Robust?
by: Zhang, Lingze, et al.
Published: (2025)

Beyond Multiple Choice: Evaluating Steering Vectors for Summarization
by: Braun, Joschka, et al.
Published: (2025)

Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
by: Lu, Meng, et al.
Published: (2025)

How Do Vision-Language Models Process Conflicting Information Across Modalities?
by: Hua, Tianze, et al.
Published: (2025)

The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling
by: Zhang, Ruochen, et al.
Published: (2024)

I Have No Mouth, and I Must Rhyme: Uncovering Internal Phonetic Representations in LLaMA 3.2
by: McLaughlin, Oliver, et al.
Published: (2025)

Stable Anisotropic Regularization
by: Rudman, William, et al.
Published: (2023)

From Memorization to Reasoning in the Spectrum of Loss Curvature
by: Merullo, Jack, et al.
Published: (2025)

Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models
by: Karkada, Dhruva, et al.
Published: (2025)

How Do Language Models Compose Functions?
by: Khandelwal, Apoorv, et al.
Published: (2025)

Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
by: Jobanputra, Mayank, et al.
Published: (2025)

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
by: Zhang, Mozhi, et al.
Published: (2025)

Persona Vectors: Monitoring and Controlling Character Traits in Language Models
by: Chen, Runjin, et al.
Published: (2025)

Language Models are Symbolic Learners in Arithmetic
by: Deng, Chunyuan, et al.
Published: (2024)

Steering Language Models with Weight Arithmetic
by: Fierro, Constanza, et al.
Published: (2025)

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
by: Lewis, Martha, et al.
Published: (2022)

Interpretable Syntactic Representations Enable Hierarchical Word Vectors
by: Silwal, Biraj
Published: (2024)

Source-Modality Monitoring in Vision-Language Models
by: Hua, Etha Tianze, et al.
Published: (2026)

From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures
by: Rottach, Florian, et al.
Published: (2025)

Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
by: Boppana, Siddharth, et al.
Published: (2026)

Unraveling Arithmetic in Large Language Models: The Role of Algebraic Structures
by: Chang, Fu-Chieh, et al.
Published: (2024)

What is an "Abstract Reasoner"? Revisiting Experiments and Arguments about Large Language Models
by: Yun, Tian, et al.
Published: (2025)

Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences
by: Sulc, Antonin
Published: (2025)

Function Vectors in Large Language Models
by: Todd, Eric, et al.
Published: (2023)

StyleBench: Evaluating thinking styles in Large Language Models
by: Guo, Junyu, et al.
Published: (2025)

Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks
by: Gambardella, Andrew, et al.
Published: (2024)

Can LLMs subtract numbers?
by: Jobanputra, Mayank, et al.
Published: (2025)

Adaptive Task Vectors for Large Language Models
by: Kang, Joonseong, et al.
Published: (2025)

Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
by: Wei, Linye, et al.
Published: (2025)

Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance
by: Esfandiarpoor, Reza, et al.
Published: (2025)

Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding
by: Cheshmi, Seyyed Saeid, et al.
Published: (2026)

Word Embeddings Are Steers for Language Models
by: Han, Chi, et al.
Published: (2023)

Understanding (Un)Reliability of Steering Vectors in Language Models
by: Braun, Joschka, et al.
Published: (2025)

Shared Lexical Task Representations Explain Behavioral Variability In LLMs
by: Yang, Zhuonan, et al.
Published: (2026)

Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models
by: Chen, Catherine, et al.
Published: (2024)