Image de couverture de livre

You Do Not Fully Utilize Transformer's Representation Capacity

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gerasimov, Gleb, Aksenov, Yaroslav, Balagansky, Nikita, Sinii, Viacheslav, Gavrilov, Daniil
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Teach Old SAEs New Domain Tricks with Boosting
par: Koriagin, Nikita, et autres
Publié: (2025)

Analyze Feature Flow to Enhance Interpretation and Steering in Language Models
par: Laptev, Daniil, et autres
Publié: (2025)

Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders
par: Kurochkin, Vadim, et autres
Publié: (2025)

Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectors
par: Sinii, Viacheslav, et autres
Publié: (2025)

Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy
par: Balagansky, Nikita, et autres
Publié: (2025)

Linear Transformers with Learnable Kernel Functions are Better In-Context Models
par: Aksenov, Yaroslav, et autres
Publié: (2024)

Learn Your Reference Model for Real Good Alignment
par: Gorbatovski, Alexey, et autres
Publié: (2024)

Diffusion Language Models Generation Can Be Halted Early
par: Vaina, Sofia Maria Lo Cicero, et autres
Publié: (2023)

Steering LLM Reasoning Through Bias-Only Adaptation
par: Sinii, Viacheslav, et autres
Publié: (2025)

Mechanistic Permutability: Match Features Across Layers
par: Balagansky, Nikita, et autres
Publié: (2024)

The Differences Between Direct Alignment Algorithms are a Blur
par: Gorbatovski, Alexey, et autres
Publié: (2025)

Next Embedding Prediction Makes World Models Stronger
par: Bredis, George, et autres
Publié: (2026)

Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success
par: Bredis, George, et autres
Publié: (2025)

F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
par: Plyusov, Daniil, et autres
Publié: (2026)

Guided Star-Shaped Masked Diffusion
par: Meshchaninov, Viacheslav, et autres
Publié: (2025)

Trust-Region Behavior Blending for On-Policy Distillation
par: Plyusov, Daniil, et autres
Publié: (2026)

ESSA: Evolutionary Strategies for Scalable Alignment
par: Korotyshova, Daria, et autres
Publié: (2025)

Exploring the Hidden Capacity of LLMs for One-Step Text Generation
par: Mezentsev, Gleb, et autres
Publié: (2025)

On the Representational Capacity of Recurrent Neural Language Models
par: Nowak, Franz, et autres
Publié: (2023)

Text-Based Detection of On-Hold Scripts in Contact Center Calls
par: Galimzianov, Dmitrii, et autres
Publié: (2024)

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions
par: Pezeshkpour, Pouya, et autres
Publié: (2024)

Disentangling the Roles of Representation and Selection in Data Pruning
par: Du, Yupei, et autres
Publié: (2025)

Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation
par: Li, Zhuohang, et autres
Publié: (2024)

Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers for Long Contexts
par: Sivtsov, Danil, et autres
Publié: (2025)

MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment
par: Hong, Dang Nguyen, et autres
Publié: (2026)

A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs
par: Goel, Raghavv, et autres
Publié: (2026)

Attention Is Not All You Need: The Importance of Feedforward Networks in Transformer Models
par: Gerber, Isaac
Publié: (2025)

Transformer See, Transformer Do: Copying as an Intermediate Step in Learning Analogical Reasoning
par: Hellwig, Philipp, et autres
Publié: (2026)

Probabilistic Topic Modelling with Transformer Representations
par: Reuter, Arik, et autres
Publié: (2024)

Transformers with Selective Access to Early Representations
par: Gunasekaran, Skye, et autres
Publié: (2026)

Discrete Neural Algorithmic Reasoning
par: Rodionov, Gleb, et autres
Publié: (2024)

Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both
par: Nath, Abhijnan, et autres
Publié: (2024)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Emergent Stack Representations in Modeling Counter Languages Using Transformers
par: Tiwari, Utkarsh, et autres
Publié: (2025)

Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?
par: Ilyankou, Ilya, et autres
Publié: (2024)

Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning
par: Arefin, Md Rifat, et autres
Publié: (2024)

Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task
par: Curth, Alicia, et autres
Publié: (2026)

Think Before You Act: Decision Transformers with Working Memory
par: Kang, Jikun, et autres
Publié: (2023)

Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers
par: Barron, Joshua, et autres
Publié: (2025)

Review, Remask, Refine (R3): Process-Guided Block Diffusion for Text Generation
par: Mounier, Nikita, et autres
Publié: (2025)