Cover Image

Emergence of a High-Dimensional Abstraction Phase in Language Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cheng, Emily, Doimo, Diego, Kervadec, Corentin, Macocco, Iuri, Yu, Jade, Laio, Alessandro, Baroni, Marco
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Tracing Computation Density in LLMs
by: Kervadec, Corentin, et al.
Published: (2026)

Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
by: Kervadec, Corentin, et al.
Published: (2026)

Prediction hubs are context-informed frequent tokens in LLMs
by: Nielsen, Beatrix M. G., et al.
Published: (2025)

Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models
by: Macocco, Iuri, et al.
Published: (2025)

Evil twins are not that evil: Qualitative insights into machine-generated prompts
by: Rakotonirina, Nathanaël Carraz, et al.
Published: (2024)

Scale-adaptive and robust intrinsic dimension estimation via optimal neighbourhood identification
by: Di Noia, Antonio, et al.
Published: (2024)

A quantitative analysis of semantic information in deep representations of text and images
by: Acevedo, Santiago, et al.
Published: (2025)

Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models
by: Cheng, Emily, et al.
Published: (2024)

The representation landscape of few-shot learning and fine-tuning in large language models
by: Doimo, Diego, et al.
Published: (2024)

Abstraction Induces the Brain Alignment of Language and Speech Models
by: Cheng, Emily, et al.
Published: (2026)

Head Pursuit: Probing Attention Specialization in Multimodal Transformers
by: Basile, Lorenzo, et al.
Published: (2025)

Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
by: Ortu, Francesco, et al.
Published: (2024)

Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations
by: Baroni, Marco, et al.
Published: (2026)

A distributional simplicity bias in the learning dynamics of transformers
by: Rende, Riccardo, et al.
Published: (2024)

Stereotypical gender actions can be extracted from Web text
by: Herdağdelen, Amaç, et al.
Published: (2025)

MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Models
by: Rakotonirina, Nathanaël Carraz, et al.
Published: (2024)

Mapping of attention mechanisms to a generalized Potts model
by: Rende, Riccardo, et al.
Published: (2023)

Abstraction-of-Thought Makes Language Models Better Reasoners
by: Hong, Ruixin, et al.
Published: (2024)

Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
by: Ranaldi, Leonardo, et al.
Published: (2025)

Functional Abstraction of Knowledge Recall in Large Language Models
by: Wang, Zijian, et al.
Published: (2025)

Certifying Phase Abstraction
by: Froleyks, Nils, et al.
Published: (2024)

The Abstraction Gap in Vision-Language Causal Reasoning
by: Hoang, Chinh, et al.
Published: (2026)

Efficient Tool Use with Chain-of-Abstraction Reasoning
by: Gao, Silin, et al.
Published: (2024)

Controllable Abstraction in Summary Generation for Large Language Models via Prompt Engineering
by: Song, Xiangchen, et al.
Published: (2025)

High-Dimensional Interlingual Representations of Large Language Models
by: Wilie, Bryan, et al.
Published: (2025)

An unsupervised tour through the hidden pathways of deep neural networks
by: Doimo, Diego
Published: (2025)

TRACE for Tracking the Emergence of Semantic Representations in Transformers
by: Aljaafari, Nura, et al.
Published: (2025)

Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
by: Zheng, Huaixiu Steven, et al.
Published: (2023)

PACE: Procedural Abstractions for Communicating Efficiently
by: Thomas, Jonathan D., et al.
Published: (2024)

Curse of High Dimensionality Issue in Transformer for Long-context Modeling
by: Zhang, Shuhai, et al.
Published: (2025)

Optimizing FDTD Solvers for Electromagnetics: A Compiler-Guided Approach with High-Level Tensor Abstractions
by: He, Yifei, et al.
Published: (2025)

Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training
by: Rahmati, Elnaz, et al.
Published: (2026)

AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
by: Wang, Zhaowei, et al.
Published: (2023)

Heterogeneous Encoders Scaling In The Transformer For Neural Machine Translation
by: Hu, Jia Cheng, et al.
Published: (2023)

Traditional Chinese Medicine Case Analysis System for High-Level Semantic Abstraction: Optimized with Prompt and RAG
by: Xu, Peng, et al.
Published: (2024)

Linearly Controlled Language Generation with Performative Guarantees
by: Cheng, Emily, et al.
Published: (2024)

Reasoning Capabilities and Invariability of Large Language Models
by: Raganato, Alessandro, et al.
Published: (2025)

Detecting Conceptual Abstraction in LLMs
by: Regneri, Michaela, et al.
Published: (2024)

PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction
by: Yu, Simon, et al.
Published: (2025)

Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction
by: Opiełka, Gustaw, et al.
Published: (2025)