Image de couverture de livre

Behavioral Steering in a 35B MoE Language Model via SAE-Decoded Probe Vectors: One Agency Axis, Not Five Traits

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Yap, Jia Qing
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Steering MoE LLMs via Expert (De)Activation
par: Fayyaz, Mohsen, et autres
Publié: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
par: Xia, Xinfeng, et autres
Publié: (2025)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
par: Huang, Haiduo, et autres
Publié: (2025)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
par: Manzoni, Andrea
Publié: (2026)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
par: Li, Zichong, et autres
Publié: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
par: Sun, Libo, et autres
Publié: (2026)

PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
par: Koromilas, Panagiotis, et autres
Publié: (2026)

LocMoE: A Low-Overhead MoE for Large Language Model Training
par: Li, Jing, et autres
Publié: (2024)

Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
par: Kang, Junmo, et autres
Publié: (2024)

VSPO: Vector-Steered Policy Optimization for Behavioral Control
par: Zhang, Xuechen, et autres
Publié: (2026)

Faster MoE LLM Inference for Extremely Large Models
par: Yang, Haoqi, et autres
Publié: (2025)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
par: Chen, Guanjie, et autres
Publié: (2024)

Persona Vectors: Monitoring and Controlling Character Traits in Language Models
par: Chen, Runjin, et autres
Publié: (2025)

FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
par: Kang, Hao, et autres
Publié: (2025)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization
par: Cao, Yuanpu, et autres
Publié: (2024)

CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
par: Xu, Yuzhuang, et autres
Publié: (2025)

DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment
par: Wedgwood, James, et autres
Publié: (2026)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
par: Huang, Quzhe, et autres
Publié: (2024)

COLD-Steer: Steering Large Language Models via In-Context One-step Learning Dynamics
par: Sharma, Kartik, et autres
Publié: (2026)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
par: Li, Shuhuai, et autres
Publié: (2026)

Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures
par: Bronzini, Marco, et autres
Publié: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
par: Takashiro, Shota, et autres
Publié: (2026)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
par: Lee, Jaeseong, et autres
Publié: (2024)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
par: Xu, Zukang, et autres
Publié: (2026)

MESA: Improving MoE Safety Alignment via Decentralized Expertise
par: Sun, Yitong, et autres
Publié: (2026)

Mixture of Tokens: Continuous MoE through Cross-Example Aggregation
par: Antoniak, Szymon, et autres
Publié: (2023)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
par: Shi, Chufan, et autres
Publié: (2024)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
par: Li, Pingzhi, et autres
Publié: (2025)

Towards an empirical understanding of MoE design choices
par: Fan, Dongyang, et autres
Publié: (2024)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
par: Deng, Jianing, et autres
Publié: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
par: Pióro, Maciej, et autres
Publié: (2024)

Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
par: Wang, Bo, et autres
Publié: (2026)

MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
par: Hannah, Lauren. A, et autres
Publié: (2025)

Predicting Where Steering Vectors Succeed
par: Billa, Jayadev
Publié: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
par: Xie, Yanyue, et autres
Publié: (2024)

Post-Trained MoE Can Skip Half Experts via Self-Distillation
par: Lv, Xingtai, et autres
Publié: (2026)

EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE
par: Chen, Junyi, et autres
Publié: (2023)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
par: Cao, Mingyu, et autres
Publié: (2024)