Image de couverture de livre

SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Fillingham, Sean P., Gordon, Andrew, Lai, Peter, Poncini, Xavier, Quarel, David, Heimersheim, Stefan
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Evolution of SAE Features Across Layers in LLMs
par: Balcells, Daniel, et autres
Publié: (2024)

Evaluating Synthetic Activations composed of SAE Latents in GPT-2
par: Giglemiani, Giorgi, et autres
Publié: (2024)

You can remove GPT2's LayerNorm by fine-tuning
par: Heimersheim, Stefan
Publié: (2024)

How to use and interpret activation patching
par: Heimersheim, Stefan, et autres
Publié: (2024)

Investigating Sensitive Directions in GPT-2: An Improved Baseline and Comparative Analysis of SAEs
par: Lee, Daniel J., et autres
Publié: (2024)

Interpreting Reinforcement Learning Agents with Susceptibilities
par: Elliott, Chris, et autres
Publié: (2026)

Characterizing stable regions in the residual stream of LLMs
par: Janiak, Jett, et autres
Publié: (2024)

SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning
par: Abbas, Farwa, et autres
Publié: (2025)

Benchmarking Deception Probes via Black-to-White Performance Boosts
par: Parrack, Avi, et autres
Publié: (2025)

Stagewise Reinforcement Learning and the Geometry of the Regret Landscape
par: Elliott, Chris, et autres
Publié: (2026)

Detecting Strategic Deception Using Linear Probes
par: Goldowsky-Dill, Nicholas, et autres
Publié: (2025)

The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
par: Taufeeque, Mohammad, et autres
Publié: (2026)

SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Grounding
par: Zabounidis, Renos, et autres
Publié: (2026)

Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
par: Baroni, Luca, et autres
Publié: (2025)

Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
par: Braun, Dan, et autres
Publié: (2025)

DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking
par: Wiemann, Matt L., et autres
Publié: (2026)

The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
par: Bushnaq, Lucius, et autres
Publié: (2024)

SCALAR: Quantifying Structural Hallucination, Consistency, and Reasoning Gaps in Materials Foundation Models
par: Polat, Can, et autres
Publié: (2026)

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
par: Yin, Lu, et autres
Publié: (2023)

Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks
par: Pandaram, Muthukumar, et autres
Publié: (2025)

Tokenized SAEs: Disentangling SAE Reconstructions
par: Dooms, Thomas, et autres
Publié: (2025)

Evaluating SAE interpretability without explanations
par: Paulo, Gonçalo, et autres
Publié: (2025)

PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
par: Koromilas, Panagiotis, et autres
Publié: (2026)

SAE: Single Architecture Ensemble Neural Networks
par: Ferianc, Martin, et autres
Publié: (2024)

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
par: Guo, Wentao, et autres
Publié: (2024)

Sparsity Forcing: Reinforcing Token Sparsity of MLLMs
par: Chen, Feng, et autres
Publié: (2025)

Chatting Up Attachment: Using LLMs to Predict Adult Bonds
par: Soares, Paulo, et autres
Publié: (2024)

Resting Neurons, Active Insights: Robustifying Activation Sparsity in LLMs via Spontaneity
par: Xu, Haotian, et autres
Publié: (2025)

Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization
par: Li, Guanchen, et autres
Publié: (2025)

SAU: Sparsity-Aware Unlearning for LLMs via Gradient Masking and Importance Redistribution
par: Wang, Yuze, et autres
Publié: (2026)

PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
par: Hourri, Younes, et autres
Publié: (2025)

Beyond Toy Benchmarks: A Systematic Evaluation of OOD Detection Methods For Plant Pathology Classification
par: Shah, Devesh
Publié: (2026)

Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
par: Bushnaq, Lucius, et autres
Publié: (2024)

MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on LLMs
par: Sun, Yan, et autres
Publié: (2025)

DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs
par: Yin, Ruokai, et autres
Publié: (2025)

OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features
par: Korznikov, Anton, et autres
Publié: (2025)

Concept-SAE: Active Causal Probing of Visual Model Behavior
par: Ding, Jianrong, et autres
Publié: (2025)

Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
par: Cao, Tue M., et autres
Publié: (2026)

Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference
par: Mukherji, Rishav, et autres
Publié: (2023)

PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models
par: Wnag, Zining, et autres
Publié: (2024)