Cover Image

Shared Global and Local Geometry of Language Model Embeddings

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Andrew, Weber, Melanie, Viégas, Fernanda, Wattenberg, Martin
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner
by: Li, Kenneth, et al.
Published: (2024)

Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
by: Lee, Andrew, et al.
Published: (2026)

Chronotome: Real-Time Topic Modeling for Streaming Embedding Spaces
by: Lim, Matte, et al.
Published: (2025)

When Bad Data Leads to Good Models
by: Li, Kenneth, et al.
Published: (2025)

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
by: Li, Kenneth, et al.
Published: (2023)

Measuring and Controlling Instruction (In)Stability in Language Model Dialogs
by: Li, Kenneth, et al.
Published: (2024)

Story Ribbons: Reimagining Storyline Visualizations with Large Language Models
by: Yeh, Catherine, et al.
Published: (2025)

The Geometry of Self-Verification in a Task-Specific Reasoning Model
by: Lee, Andrew, et al.
Published: (2025)

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
by: Li, Kenneth, et al.
Published: (2022)

Decomposing Query-Key Feature Interactions Using Contrastive Covariances
by: Lee, Andrew, et al.
Published: (2026)

What Does it Mean for a Neural Network to Learn a "World Model"?
by: Li, Kenneth, et al.
Published: (2025)

Relational Composition in Neural Networks: A Survey and Call to Action
by: Wattenberg, Martin, et al.
Published: (2024)

Including Facial Expressions in Contextual Embeddings for Sign Language Generation
by: Viegas, Carla, et al.
Published: (2022)

Global-to-Local Support Spectrums for Language Model Explainability
by: Agussurja, Lucas, et al.
Published: (2024)

Geometry-Preserving Aggregation for Mixture-of-Experts Embedding Models
by: Kachuee, Sajjad, et al.
Published: (2026)

Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings
by: Patil, Sarang, et al.
Published: (2025)

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
by: Wang, Jingcun, et al.
Published: (2024)

Embedding Geometries of Contrastive Language-Image Pre-Training
by: Chou, Jason Chuan-Chih, et al.
Published: (2024)

From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
by: Kiruluta, Andrew
Published: (2026)

From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Models
by: Wang, Ziyan, et al.
Published: (2025)

Sharing Parameter by Conjugation for Knowledge Graph Embeddings in Complex Space
by: Feng, Xincan, et al.
Published: (2024)

ICLR: In-Context Learning of Representations
by: Park, Core Francisco, et al.
Published: (2024)

Scaling Embedding Layers in Language Models
by: Yu, Da, et al.
Published: (2025)

Repetition Improves Language Model Embeddings
by: Springer, Jacob Mitchell, et al.
Published: (2024)

Geometry-Calibrated Conformal Abstention for Language Models
by: Xu, Rui, et al.
Published: (2026)

The Geometry of Tokens in Internal Representations of Large Language Models
by: Viswanathan, Karthik, et al.
Published: (2025)

Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
by: Lim, Sungjun, et al.
Published: (2026)

Block Transformer: Global-to-Local Language Modeling for Fast Inference
by: Ho, Namgyu, et al.
Published: (2024)

Spherical Steering: Geometry-Aware Activation Rotation for Language Models
by: You, Zejia, et al.
Published: (2026)

Can Interpretation Predict Behavior on Unseen Data?
by: Li, Victoria R., et al.
Published: (2025)

DEPT: Decoupled Embeddings for Pre-training Language Models
by: Iacob, Alex, et al.
Published: (2024)

Language Model Training Paradigms for Clinical Feature Embeddings
by: Hu, Yurong, et al.
Published: (2023)

Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism
by: Orgad, Hadas, et al.
Published: (2026)

EmbedLLM: Learning Compact Representations of Large Language Models
by: Zhuang, Richard, et al.
Published: (2024)

Foundation Model's Embedded Representations May Detect Distribution Shift
by: Vargas, Max, et al.
Published: (2023)

Scale Determines Whether Language Models Organize Representation Geometry for Prediction
by: Xu, Weilun
Published: (2026)

Integrating Locality-Aware Attention with Transformers for General Geometry PDEs
by: Koh, Minsu, et al.
Published: (2025)

Geometry of Decision Making in Language Models
by: Joshi, Abhinav, et al.
Published: (2025)

Demystifying Embedding Spaces using Large Language Models
by: Tennenholtz, Guy, et al.
Published: (2023)

Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence
by: Nava, Andres, et al.
Published: (2026)