Cover Image

Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Honda, Ukyo, Oka, Tatsushi, Zhang, Peinan, Mita, Masato
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring Explanations Improves the Robustness of In-Context Learning
by: Honda, Ukyo, et al.
Published: (2025)

FaithCAMERA: Construction of a Faithful Dataset for Ad Text Generation
by: Kato, Akihiko, et al.
Published: (2024)

Annotation-Efficient Language Model Alignment via Diverse and Representative Response Texts
by: Jinnai, Yuu, et al.
Published: (2024)

Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?
by: Hoshino, Sho, et al.
Published: (2026)

Distilling Many-Shot In-Context Learning into a Cheat Sheet
by: Honda, Ukyo, et al.
Published: (2025)

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective
by: Kajitsuka, Tokio, et al.
Published: (2026)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
by: Cai, Weilin, et al.
Published: (2024)

Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding
by: Jinnai, Yuu, et al.
Published: (2024)

A Single Linear Layer Yields Task-Adapted Low-Rank Matrices
by: Kim, Hwichan, et al.
Published: (2024)

Geometry-Preserving Aggregation for Mixture-of-Experts Embedding Models
by: Kachuee, Sajjad, et al.
Published: (2026)

Bayesian Mixture of Experts For Large Language Models
by: Dialameh, Maryam, et al.
Published: (2025)

Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation
by: Mita, Masato, et al.
Published: (2023)

Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations
by: Bhan, Milan, et al.
Published: (2024)

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models
by: Yuan, Yu, et al.
Published: (2024)

Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
by: Ying, Jiahao, et al.
Published: (2025)

When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
by: Yoon, Youngsik, et al.
Published: (2026)

A Survey on Mixture of Experts in Large Language Models
by: Cai, Weilin, et al.
Published: (2024)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

HMoE: Heterogeneous Mixture of Experts for Language Modeling
by: Wang, An, et al.
Published: (2024)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models
by: Zhou, Yuqing, et al.
Published: (2024)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

A Closer Look into Mixture-of-Experts in Large Language Models
by: Lo, Ka Man, et al.
Published: (2024)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
by: Wang, Zihan, et al.
Published: (2025)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
by: Zhang, Zeliang, et al.
Published: (2024)

LCTG Bench: LLM Controlled Text Generation Benchmark
by: Kurihara, Kentaro, et al.
Published: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
by: Ma, Zhicheng, et al.
Published: (2026)

Upcycling Large Language Models into Mixture of Experts
by: He, Ethan, et al.
Published: (2024)

Toward LLMs Beyond English-Centric Development
by: Takase, Sho, et al.
Published: (2026)

MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
by: Tastan, Nurbek, et al.
Published: (2026)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
by: Liu, Zehua, et al.
Published: (2025)

QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
by: Li, Pingzhi, et al.
Published: (2024)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
by: Lv, Ang, et al.
Published: (2025)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
by: Takashiro, Shota, et al.
Published: (2026)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
by: He, Haoze, et al.
Published: (2026)

Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification
by: Eshuijs, Leon, et al.
Published: (2025)

AdTEC: A Unified Benchmark for Evaluating Text Quality in Search Engine Advertising
by: Zhang, Peinan, et al.
Published: (2024)

Mixture of Experts Made Intrinsically Interpretable
by: Yang, Xingyi, et al.
Published: (2025)

Maximum Score Routing For Mixture-of-Experts
by: Dong, Bowen, et al.
Published: (2025)