Image de couverture de livre

Layer by Layer: Uncovering Hidden Representations in Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Skean, Oscar, Arefin, Md Rifat, Zhao, Dan, Patel, Niket, Naghiyev, Jalal, LeCun, Yann, Shwartz-Ziv, Ravid
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Does Representation Matter? Exploring Intermediate Layers in Large Language Models
par: Skean, Oscar, et autres
Publié: (2024)

Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning
par: Arefin, Md Rifat, et autres
Publié: (2024)

Video Representation Learning with Joint-Embedding Predictive Architectures
par: Drozdov, Katrina, et autres
Publié: (2024)

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
par: Shani, Chen, et autres
Publié: (2025)

Learning to Compress: Local Rank and Information Compression in Deep Neural Networks
par: Patel, Niket, et autres
Publié: (2024)

Variance-Covariance Regularization Improves Representation Learning
par: Zhu, Jiachen, et autres
Publié: (2023)

AI Must Embrace Specialization via Superhuman Adaptable Intelligence
par: Goldfeder, Judah, et autres
Publié: (2026)

The Entropy Enigma: Success and Failure of Entropy Minimization
par: Press, Ori, et autres
Publié: (2024)

Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training
par: Nepal, Aadim, et autres
Publié: (2025)

On Training in Imagination
par: Timor, Nadav, et autres
Publié: (2026)

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
par: Sanyal, Sunny, et autres
Publié: (2024)

You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
par: LeVi, Amit, et autres
Publié: (2025)

JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention
par: Ioannides, Georgios, et autres
Publié: (2025)

LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures
par: Huang, Hai, et autres
Publié: (2025)

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization
par: Shwartz-Ziv, Ravid, et autres
Publié: (2023)

Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation
par: Zeevi, Tal, et autres
Publié: (2024)

Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
par: Paech, Samuel, et autres
Publié: (2025)

Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
par: Ioannides, Georgios, et autres
Publié: (2026)

Just How Flexible are Neural Networks in Practice?
par: Shwartz-Ziv, Ravid, et autres
Publié: (2024)

Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
par: Queipo-de-Llano, Enrique, et autres
Publié: (2025)

Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
par: Rofin, Mark, et autres
Publié: (2026)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
par: Sun, Shangwen, et autres
Publié: (2026)

Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence
par: Dawid, Anna, et autres
Publié: (2023)

Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
par: Chen, Angelica, et autres
Publié: (2023)

Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations
par: Mialon, Grégoire, et autres
Publié: (2022)

Fast and Exact Enumeration of Deep Networks Partitions Regions
par: Balestriero, Randall, et autres
Publié: (2024)

LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
par: Balestriero, Randall, et autres
Publié: (2025)

Learning by Reconstruction Produces Uninformative Features For Perception
par: Balestriero, Randall, et autres
Publié: (2024)

Forgotten Polygons: Multimodal Large Language Models are Shape-Blind
par: Rudman, William, et autres
Publié: (2025)

Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
par: Zhao, Zheng, et autres
Publié: (2024)

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs
par: Nguyen, Minh Nhat, et autres
Publié: (2024)

The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs
par: Janiak, Denis, et autres
Publié: (2025)

Transformers without Normalization
par: Zhu, Jiachen, et autres
Publié: (2025)

URLOST: Unsupervised Representation Learning without Stationarity or Topology
par: Yun, Zeyu, et autres
Publié: (2023)

LiveBench: A Challenging, Contamination-Limited LLM Benchmark
par: White, Colin, et autres
Publié: (2024)

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
par: Schaeffer, Rylan, et autres
Publié: (2024)

Black-box Context-free Grammar Inference for Readable & Natural Grammars
par: Arefin, Mohammad Rifat, et autres
Publié: (2025)

Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA
par: Huang, Hai, et autres
Publié: (2026)

Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science
par: Dupoux, Emmanuel, et autres
Publié: (2026)

A hierarchical loss and its problems when classifying non-hierarchically
par: Wu, Cinna, et autres
Publié: (2017)