Cover Image

The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Razzhigaev, Anton, Mikhalchuk, Matvey, Goncharova, Elizaveta, Oseledets, Ivan, Dimitrov, Denis, Kuznetsov, Andrey
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence Information Theory Machine Learning General Topology
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Your Transformer is Secretly Linear
by: Razzhigaev, Anton, et al.
Published: (2024)

LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
by: Razzhigaev, Anton, et al.
Published: (2025)

OmniFusion Technical Report
by: Goncharova, Elizaveta, et al.
Published: (2024)

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
by: Chekalina, Viktoriia, et al.
Published: (2024)

Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
by: Li, Pengyi, et al.
Published: (2026)

Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
by: Tulchinskii, Eduard, et al.
Published: (2023)

Knot Theory and Error-Correcting Codes
by: Kilic, Altan B., et al.
Published: (2023)

Real-World Transferable Adversarial Attack on Face-Recognition Systems
by: Kaznacheev, Andrey, et al.
Published: (2025)

SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
by: Dragunov, Nikita, et al.
Published: (2025)

Logit-KL Flow Matching: Non-Autoregressive Text Generation via Sampling-Hybrid Inference
by: Sevriugov, Egor, et al.
Published: (2024)

Simple Vision-Language Math Reasoning via Rendered Text
by: Skripkin, Matvey, et al.
Published: (2025)

Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders
by: Kuznetsov, Kristian, et al.
Published: (2025)

Sentence-Anchored Gist Compression for Long-Context LLMs
by: Tarasov, Dmitrii, et al.
Published: (2025)

Universal Adversarial Attack on Aligned Multimodal LLMs
by: Rahmatullaev, Temurbek, et al.
Published: (2025)

Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space
by: Razzhigaev, Anton, et al.
Published: (2025)

Codensity: Isbell duality, pro-objects, compactness and accessibility
by: Di Liberti, Ivan
Published: (2019)

Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
by: Li, Pengyi, et al.
Published: (2025)

Strongly bounded generation in transformation groups
by: Vlamis, Nicholas G.
Published: (2025)

Tree-like is not a transitive relation on paths
by: Brazas, Jeremy, et al.
Published: (2026)

Fibrations and coset spaces for locally compact groups
by: Kramer, Linus, et al.
Published: (2024)

Whitehead doubling, rank estimate and nonembeddability of contractible open manifolds
by: Gu, Shijie, et al.
Published: (2025)

Geometric models and asymptotic dimension for infinite-type surface mapping class groups
by: Kopreski, Michael C., et al.
Published: (2025)

MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
by: Skripkin, Matvey, et al.
Published: (2025)

Homotopy Type of Intersections of Real Bruhat Cells in Dimension 6
by: Leal, Giovanna, et al.
Published: (2025)

Relatively functionally countable subsets of products
by: Lipin, Anton
Published: (2024)

On resolvability and tightness in uncountable spaces
by: Lipin, Anton
Published: (2024)

Resolvability in products of spaces of small cardinality
by: Lipin, Anton
Published: (2025)

Resolvability in products and squares
by: Lipin, Anton
Published: (2025)

The Shape of Compact Covers
by: Feng, Ziqin, et al.
Published: (2024)

The Shape of Generating Families
by: Feng, Ziqin, et al.
Published: (2025)

Exact categories of topological vector spaces with linear topology
by: Positselski, Leonid
Published: (2020)

Solvability and Rigidity for Topological Skew Braces
by: Damele, Marco, et al.
Published: (2026)

Structure Theorems for locally compact modules over localizations of the integers
by: Lourenço, Pedro
Published: (2026)

On the existence and properties of Alexandroff paratopological groups
by: Chocano, Pedro J., et al.
Published: (2026)

The lattice of smooth sublocales as a Bruns-Lakser completion
by: Arrieta, Igor, et al.
Published: (2026)

The characterization for the sobriety of $L$-convex spaces
by: Wu, Guojun, et al.
Published: (2024)

Constructing Psuedo-$τ$-fine Precompact Groups
by: Peng, Dekui, et al.
Published: (2024)

Ordered Locales
by: Heunen, Chris, et al.
Published: (2023)

The reflectivity of some categories of T0 spaces in domain theory
by: Shen, Chong, et al.
Published: (2021)

Polish topologies on endomorphism monoids of linear orders
by: Bardyla, Serhii, et al.
Published: (2026)