Cover Image

On the Origins of Linear Representations in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jiang, Yibo, Rajendran, Goutham, Ravikumar, Pradeep, Aragam, Bryon, Veitch, Victor
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
by: Jiang, Yibo, et al.
Published: (2024)

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
by: Rajendran, Goutham, et al.
Published: (2024)

Identifying General Mechanism Shifts in Linear Causal Representations
by: Chen, Tianyu, et al.
Published: (2024)

The Linear Representation Hypothesis and the Geometry of Large Language Models
by: Park, Kiho, et al.
Published: (2023)

The Geometry of Categorical and Hierarchical Concepts in Large Language Models
by: Park, Kiho, et al.
Published: (2024)

iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models
by: Chen, Tianyu, et al.
Published: (2023)

Markov Equivalence and Consistency in Differentiable Structure Learning
by: Deng, Chang, et al.
Published: (2024)

An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis
by: Rajendran, Goutham, et al.
Published: (2023)

LLM-Select: Feature Selection with Large Language Models
by: Jeong, Daniel P., et al.
Published: (2024)

Greedy equivalence search for nonparametric graphical models
by: Aragam, Bryon
Published: (2024)

BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling
by: Gui, Lin, et al.
Published: (2024)

Towards Interpretable Deep Generative Models via Causal Representation Learning
by: Moran, Gemma E., et al.
Published: (2025)

Model-free Estimation of Latent Structure via Multiscale Nonparametric Maximum Likelihood
by: Aragam, Bryon, et al.
Published: (2024)

Differentiable Structure Learning and Causal Discovery for General Binary Data
by: Deng, Chang, et al.
Published: (2025)

Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
by: Saglam, Baturay, et al.
Published: (2025)

Concept Algebra for (Score-Based) Text-Controlled Generative Models
by: Wang, Zihao, et al.
Published: (2023)

SOMP: Scalable Gradient Inversion for Large Language Models via Subspace-Guided Orthogonal Matching Pursuit
by: Li, Yibo, et al.
Published: (2026)

Bias in Large Language Models: Origin, Evaluation, and Mitigation
by: Guo, Yufei, et al.
Published: (2024)

Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representation
by: Wu, Xingyu, et al.
Published: (2023)

Optimal structure learning and conditional independence testing
by: Gao, Ming, et al.
Published: (2025)

Linear Dynamics in the RLVR Training of Large Language Models
by: Wang, Tianle, et al.
Published: (2026)

Lizard: An Efficient Linearization Framework for Large Language Models
by: Van Nguyen, Chien, et al.
Published: (2025)

Linear Representations of Political Perspective Emerge in Large Language Models
by: Kim, Junsol, et al.
Published: (2025)

The Geometry of Tokens in Internal Representations of Large Language Models
by: Viswanathan, Karthik, et al.
Published: (2025)

Graph Linearization Methods for Reasoning on Graphs with Large Language Models
by: Xypolopoulos, Christos, et al.
Published: (2024)

Beyond identifiability: Learning causal representations with few environments and finite samples
by: Lee, Inbeom, et al.
Published: (2026)

Can Large Language Models Generalize Procedures Across Representations?
by: Lin, Fangru, et al.
Published: (2026)

InnerThoughts: Disentangling Representations and Predictions in Large Language Models
by: Chételat, Didier, et al.
Published: (2025)

Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models
by: Wehner, Jan, et al.
Published: (2025)

RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
by: Shetty, Saisha Pradeep, et al.
Published: (2026)

Can Large Language Models Understand Intermediate Representations in Compilers?
by: Jiang, Hailong, et al.
Published: (2025)

Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
by: Zhu, Qi, et al.
Published: (2024)

Does Representation Matter? Exploring Intermediate Layers in Large Language Models
by: Skean, Oscar, et al.
Published: (2024)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
by: Yang, Junjie, et al.
Published: (2025)

Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models
by: Yamashita, Tomoya, et al.
Published: (2025)

Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs
by: Qin, Yijian, et al.
Published: (2023)

Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
by: Bello, Femi, et al.
Published: (2025)

Equivalent Linear Mappings of Large Language Models
by: Golden, James R.
Published: (2025)

A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization
by: Wang, Ziqing, et al.
Published: (2025)

GLBench: A Comprehensive Benchmark for Graph with Large Language Models
by: Li, Yuhan, et al.
Published: (2024)