Cover Image

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Kenneth, Hopkins, Aspen K., Bau, David, Viégas, Fernanda, Pfister, Hanspeter, Wattenberg, Martin
Format:	Preprint
Published:	2022
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
by: Li, Kenneth, et al.
Published: (2023)

Measuring and Controlling Instruction (In)Stability in Language Model Dialogs
by: Li, Kenneth, et al.
Published: (2024)

What Does it Mean for a Neural Network to Learn a "World Model"?
by: Li, Kenneth, et al.
Published: (2025)

Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner
by: Li, Kenneth, et al.
Published: (2024)

When Bad Data Leads to Good Models
by: Li, Kenneth, et al.
Published: (2025)

The Geometry of Self-Verification in a Task-Specific Reasoning Model
by: Lee, Andrew, et al.
Published: (2025)

Relational Composition in Neural Networks: A Survey and Call to Action
by: Wattenberg, Martin, et al.
Published: (2024)

Shared Global and Local Geometry of Language Model Embeddings
by: Lee, Andrew, et al.
Published: (2025)

Emergent Representations of Program Semantics in Language Models Trained on Programs
by: Jin, Charles, et al.
Published: (2023)

Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions
by: Lee, Andrew, et al.
Published: (2026)

Chronotome: Real-Time Topic Modeling for Streaming Embedding Spaces
by: Lim, Matte, et al.
Published: (2025)

Designing a Dashboard for Transparency and Control of Conversational AI
by: Chen, Yida, et al.
Published: (2024)

Impact of Task Phrasing on Presumptions in Large Language Models
by: Ong, Kenneth J. K.
Published: (2026)

Shared Lexical Task Representations Explain Behavioral Variability In LLMs
by: Yang, Zhuonan, et al.
Published: (2026)

Generative Emergent Communication: Large Language Model is a Collective World Model
by: Taniguchi, Tadahiro, et al.
Published: (2024)

Joint-Task Regularization for Partially Labeled Multi-Task Learning
by: Nishi, Kento, et al.
Published: (2024)

ICLR: In-Context Learning of Representations
by: Park, Core Francisco, et al.
Published: (2024)

A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
by: Lee, Andrew, et al.
Published: (2024)

Do explanations generalize across large reasoning models?
by: Pal, Koyena, et al.
Published: (2026)

Story Ribbons: Reimagining Storyline Visualizations with Large Language Models
by: Yeh, Catherine, et al.
Published: (2025)

Can Interpretation Predict Behavior on Unseen Data?
by: Li, Victoria R., et al.
Published: (2025)

Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality Prediction
by: Sainsbury, Chris, et al.
Published: (2026)

Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models
by: Xu, Ningyu, et al.
Published: (2026)

Discovering Forbidden Topics in Language Models
by: Rager, Can, et al.
Published: (2025)

Linearity of Relation Decoding in Transformer Language Models
by: Hernandez, Evan, et al.
Published: (2023)

Robustness as an Emergent Property of Task Performance
by: Ashury-Tahan, Shir, et al.
Published: (2026)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

The Dual-Route Model of Induction
by: Feucht, Sheridan, et al.
Published: (2025)

ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning
by: Choi, Jihye, et al.
Published: (2025)

Uncovering Emergent Physics Representations Learned In-Context by Large Language Models
by: Song, Yeongwoo, et al.
Published: (2025)

Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks
by: Pink, Mathis, et al.
Published: (2024)

Emergent Hierarchical Structure in Large Language Models: An Information-Theoretic Framework for Multi-Scale Representation
by: Zhang, Yukin, et al.
Published: (2025)

SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging
by: Pourreza, Mohammadreza, et al.
Published: (2024)

Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Function
by: Kwok, Chin Yuen, et al.
Published: (2025)

Warmup Generations: A Task-Agnostic Approach for Guiding Sequence-to-Sequence Learning with Unsupervised Initial State Generation
by: Li, Senyu, et al.
Published: (2025)

Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
by: Sahitaj, Premtim, et al.
Published: (2025)

New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models
by: Wunderle, Julia, et al.
Published: (2025)

GPTs Are Multilingual Annotators for Sequence Generation Tasks
by: Choi, Juhwan, et al.
Published: (2024)

Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism
by: Orgad, Hadas, et al.
Published: (2026)

Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data
by: Cook, John, et al.
Published: (2026)