Cover Image

Towards Understanding Distilled Reasoning Models: A Representational Approach

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Baek, David D., Tegmark, Max
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory
by: Baek, David D., et al.
Published: (2024)

Investigating Representation Universality: Case Study on Genealogical Representations
by: Baek, David D., et al.
Published: (2024)

Harmonic Loss Trains Interpretable AI Models
by: Baek, David D., et al.
Published: (2025)

Scaling Laws For Scalable Oversight
by: Engels, Joshua, et al.
Published: (2025)

Language Models Represent Space and Time
by: Gurnee, Wes, et al.
Published: (2023)

Language Models Use Trigonometry to Do Addition
by: Kantamneni, Subhash, et al.
Published: (2025)

Low-Rank Adapting Models for Sparse Autoencoders
by: Chen, Matthew, et al.
Published: (2025)

A Neural Scaling Law from Lottery Ticket Ensembling
by: Liu, Ziming, et al.
Published: (2023)

Decomposing The Dark Matter of Sparse Autoencoders
by: Engels, Joshua, et al.
Published: (2024)

Survival of the Fittest Representation: A Case Study with Modular Addition
by: Ding, Xiaoman Delores, et al.
Published: (2024)

The Geometry of Concepts: Sparse Autoencoder Feature Structure
by: Li, Yuxiao, et al.
Published: (2024)

Do Two AI Scientists Agree?
by: Fu, Xinghong, et al.
Published: (2025)

OptPDE: Discovering Novel Integrable Systems via AI-Human Collaboration
by: Kantamneni, Subhash, et al.
Published: (2024)

A Resource Model For Neural Scaling Law
by: Song, Jinyeop, et al.
Published: (2024)

On the creation of narrow AI: hierarchy and nonlocality of neural network skills
by: Michaud, Eric J., et al.
Published: (2025)

Not All Language Model Features Are One-Dimensionally Linear
by: Engels, Joshua, et al.
Published: (2024)

The Quantization Model of Neural Scaling
by: Michaud, Eric J., et al.
Published: (2023)

How Do Transformers "Do" Physics? Investigating the Simple Harmonic Oscillator
by: Kantamneni, Subhash, et al.
Published: (2024)

Neural Thermodynamic Laws for Large Language Model Training
by: Liu, Ziming, et al.
Published: (2025)

Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
by: Kantamneni, Subhash, et al.
Published: (2025)

Towards Understanding the Shape of Representations in Protein Language Models
by: Beshkov, Kosio, et al.
Published: (2025)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
by: Kim, Minsang, et al.
Published: (2026)

The Remarkable Robustness of LLMs: Stages of Inference?
by: Lad, Vedang, et al.
Published: (2024)

Efficient Dictionary Learning with Switch Sparse Autoencoders
by: Mudide, Anish, et al.
Published: (2024)

Towards Understanding Self-play for LLM Reasoning
by: Chae, Justin Yang, et al.
Published: (2025)

Towards Understanding the Benefit of Multitask Representation Learning in Decision Process
by: Lu, Rui, et al.
Published: (2025)

What Do GNNs Actually Learn? Towards Understanding their Representations
by: Nikolentzos, Giannis, et al.
Published: (2023)

Self-Supervised On-Policy Distillation for Reasoning Language Models
by: Tan, Zhiquan, et al.
Published: (2026)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
by: Zhao, Siyan, et al.
Published: (2026)

Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
by: Chen, Danchun, et al.
Published: (2026)

Physics of Skill Learning
by: Liu, Ziming, et al.
Published: (2025)

Towards Sparse Video Understanding and Reasoning
by: Xu, Chenwei, et al.
Published: (2026)

KAN 2.0: Kolmogorov-Arnold Networks Meet Science
by: Liu, Ziming, et al.
Published: (2024)

On Learning Representations for Tabular Data Distillation
by: Kang, Inwon, et al.
Published: (2025)

Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models
by: Yin, Huifeng, et al.
Published: (2025)

Towards a Physics Foundation Model
by: Wiesner, Florian, et al.
Published: (2025)

Weight Ensembling Improves Reasoning in Language Models
by: Dang, Xingyu, et al.
Published: (2025)

Reverse Distillation: Consistently Scaling Protein Language Model Representations
by: Catrina, Darius, et al.
Published: (2026)

Similarity of Neural Network Models: A Survey of Functional and Representational Measures
by: Klabunde, Max, et al.
Published: (2023)

SPREAD: Subspace Representation Distillation for Lifelong Imitation Learning
by: Roy, Kaushik, et al.
Published: (2026)