Image de couverture de livre

Mixture of Experts Made Intrinsically Interpretable

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Xingyi, Venhoff, Constantin, Khakzar, Ashkan, de Witt, Christian Schroeder, Dokania, Puneet K., Bibi, Adel, Torr, Philip
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval
par: Venhoff, Constantin, et autres
Publié: (2025)

Towards Understanding Multimodal Fine-Tuning: Spatial Features
par: Naghashyar, Lachin, et autres
Publié: (2026)

How Visual Representations Map to Language Feature Space in Multimodal LLMs
par: Venhoff, Constantin, et autres
Publié: (2025)

SAGE: Scalable Ground Truth Evaluations for Large Sparse Autoencoders
par: Venhoff, Constantin, et autres
Publié: (2024)

Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation
par: Eiras, Francisco, et autres
Publié: (2023)

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations
par: Petrov, Aleksandar, et autres
Publié: (2023)

Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders
par: Lan, Michael, et autres
Publié: (2024)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
par: Zhang, Wenxuan, et autres
Publié: (2024)

What Makes and Breaks Safety Fine-tuning? A Mechanistic Study
par: Jain, Samyak, et autres
Publié: (2024)

Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models
par: Eiras, Francisco, et autres
Publié: (2024)

Fine-tuning can cripple your foundation model; preserving features may be the solution
par: Mukhoti, Jishnu, et autres
Publié: (2023)

Universal In-Context Approximation By Prompting Fully Recurrent Models
par: Petrov, Aleksandar, et autres
Publié: (2024)

Understanding Reasoning in Thinking Language Models via Steering Vectors
par: Venhoff, Constantin, et autres
Publié: (2025)

Base Models Know How to Reason, Thinking Models Learn When
par: Venhoff, Constantin, et autres
Publié: (2025)

AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction
par: Sesodia, Magnus, et autres
Publié: (2025)

Prompting a Pretrained Transformer Can Be a Universal Approximator
par: Petrov, Aleksandar, et autres
Publié: (2024)

DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detection
par: Aljaafari, Tala, et autres
Publié: (2025)

Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
par: Hammoud, Hasan Abed Al Kader, et autres
Publié: (2024)

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation
par: Yang, Yibo, et autres
Publié: (2024)

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability
par: Rose, Aaron, et autres
Publié: (2026)

Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
par: Lan, Michael, et autres
Publié: (2023)

MoDification: Mixture of Depths Made Easy
par: Zhang, Chen, et autres
Publié: (2024)

Latent Guard: a Safety Framework for Text-to-image Generation
par: Liu, Runtao, et autres
Publié: (2024)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
par: Wang, Haoxiang, et autres
Publié: (2024)

AttributionLab: Faithfulness of Feature Attribution Under Controllable Environments
par: Zhang, Yang, et autres
Publié: (2023)

Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models
par: Luo, Jun, et autres
Publié: (2024)

Beyond Linear Probes: Dynamic Safety Monitoring for Language Models
par: Oldfield, James, et autres
Publié: (2025)

Random Representations Outperform Online Continually Learned Representations
par: Prabhu, Ameya, et autres
Publié: (2024)

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
par: Udandarao, Vishaal, et autres
Publié: (2024)

Detecting LLM Hallucination Through Layer-wise Information Deficiency: Analysis of Ambiguous Prompts and Unanswerable Questions
par: Kim, Hazel, et autres
Publié: (2024)

Towards Certification of Uncertainty Calibration under Adversarial Attacks
par: Emde, Cornelius, et autres
Publié: (2024)

Mixture of Lookup Experts
par: Jie, Shibo, et autres
Publié: (2025)

The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level
par: Herbst, Jeremy, et autres
Publié: (2026)

Toward Robust Real-World Audio Deepfake Detection: Closing the Explainability Gap
par: Channing, Georgia, et autres
Publié: (2024)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
par: Huang, Wei, et autres
Publié: (2024)

Shh, don't say that! Domain Certification in LLMs
par: Emde, Cornelius, et autres
Publié: (2025)

Minimalist Concept Erasure in Generative Models
par: Zhang, Yang, et autres
Publié: (2025)

Multilingual Routing in Mixture-of-Experts
par: Bandarkar, Lucas, et autres
Publié: (2025)

OpenSanctions Pairs: Large-Scale Entity Matching with LLMs
par: Smith, Chandler, et autres
Publié: (2026)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
par: Zheng, Chen, et autres
Publié: (2025)