Copertina

Steering MoE LLMs via Expert (De)Activation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Fayyaz, Mohsen, Modarressi, Ali, Deilamsalehy, Hanieh, Dernoncourt, Franck, Rossi, Ryan, Bui, Trung, Schütze, Hinrich, Peng, Nanyun
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

NoLiMa: Long-Context Evaluation Beyond Literal Matching
di: Modarressi, Ali, et al.
Pubblicazione: (2025)

Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence
di: Fayyaz, Mohsen, et al.
Pubblicazione: (2025)

Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs
di: Parmar, Mihir, et al.
Pubblicazione: (2024)

RET-LLM: Towards a General Read-Write Memory for Large Language Models
di: Modarressi, Ali, et al.
Pubblicazione: (2023)

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory
di: Modarressi, Ali, et al.
Pubblicazione: (2024)

Consistent Document-Level Relation Extraction via Counterfactuals
di: Modarressi, Ali, et al.
Pubblicazione: (2024)

Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models
di: Nguyen, Minh, et al.
Pubblicazione: (2024)

With Argus Eyes: Assessing Retrieval Gaps via Uncertainty Scoring to Detect and Remedy Retrieval Blind Spots
di: Taghavi, Zeinab Sadat, et al.
Pubblicazione: (2026)

Scaling Up Video Summarization Pretraining with Large Language Models
di: Argaw, Dawit Mureja, et al.
Pubblicazione: (2024)

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing
di: Zhao, Raoyuan, et al.
Pubblicazione: (2025)

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
di: Kargaran, Amir Hossein, et al.
Pubblicazione: (2024)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models
di: Hakimi, Ahmad Dawar, et al.
Pubblicazione: (2025)

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge
di: Taghavi, Zeinab Sadat, et al.
Pubblicazione: (2025)

The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
di: Cao, Yuan, et al.
Pubblicazione: (2026)

Blind to the Human Touch: Overlap Bias in LLM-Based Summary Evaluation
di: Fang, Jiangnan, et al.
Pubblicazione: (2026)

Taipan: Efficient and Expressive State Space Language Models with Selective Attention
di: Van Nguyen, Chien, et al.
Pubblicazione: (2024)

CORG: Generating Answers from Complex, Interrelated Contexts
di: Lee, Hyunji, et al.
Pubblicazione: (2025)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
di: Wu, Haoyuan, et al.
Pubblicazione: (2025)

Evaluating Human Alignment and Model Faithfulness of LLM Rationale
di: Fayyaz, Mohsen, et al.
Pubblicazione: (2024)

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
di: Gallegos, Isabel O., et al.
Pubblicazione: (2024)

Lizard: An Efficient Linearization Framework for Large Language Models
di: Van Nguyen, Chien, et al.
Pubblicazione: (2025)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
di: Zhao, Ziyu, et al.
Pubblicazione: (2026)

Steer-MoE: Efficient Audio-Language Alignment with a Mixture-of-Experts Steering Module
di: Feng, Ruitao, et al.
Pubblicazione: (2025)

Expert Routing for Communication-Efficient MoE via Finite Expert Banks
di: Salehi, Mohammad Reza Deylam, et al.
Pubblicazione: (2026)

Exploiting the Experts: Unauthorized Compression in MoE-LLMs
di: Neogi, Pinaki Prasad Guha, et al.
Pubblicazione: (2025)

A Multi-LLM Debiasing Framework
di: Owens, Deonna M., et al.
Pubblicazione: (2024)

MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
di: Chen, Xiaodong, et al.
Pubblicazione: (2025)

PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
di: Goswami, Kanika, et al.
Pubblicazione: (2025)

PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents
di: Goswami, Kanika, et al.
Pubblicazione: (2025)

Personalized Graph-Based Retrieval for Large Language Models
di: Au, Steven, et al.
Pubblicazione: (2025)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
di: Li, Lujun, et al.
Pubblicazione: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Models
di: Hu, Guimin, et al.
Pubblicazione: (2026)

Fast MoE Inference via Predictive Prefetching and Expert Replication
di: Jyothish, Ankit, et al.
Pubblicazione: (2026)

Agentic Planning with Reasoning for Image Styling via Offline RL
di: Mukherjee, Subhojyoti, et al.
Pubblicazione: (2026)

BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
di: Wu, Juntong, et al.
Pubblicazione: (2026)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
di: Li, Pingzhi, et al.
Pubblicazione: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
di: Tran, Huyen Ngoc, et al.
Pubblicazione: (2026)