Cover Image

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Dongxu, Xu, Bing, Chen, Yinzhuo, Xu, Bufan, Lu, Wenpeng, Yang, Muyun, Zhao, Tiejun
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Self-Evaluation of Large Language Model based on Glass-box Features
by: Huang, Hui, et al.
Published: (2024)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
by: Zhou, Hongyun, et al.
Published: (2024)

LLM-based Discriminative Reasoning for Knowledge Graph Question Answering
by: Xu, Mufan, et al.
Published: (2024)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
by: Wu, Haoyuan, et al.
Published: (2025)

Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization
by: Zhou, Hongli, et al.
Published: (2026)

An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
by: Huang, Hui, et al.
Published: (2024)

A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR
by: Zheng, Yuang, et al.
Published: (2026)

Memory-augmented Query Reconstruction for LLM-based Knowledge Graph Reasoning
by: Xu, Mufan, et al.
Published: (2025)

User-Aware Active Knowledge Acquisition for Emotional Support Dialogue
by: Xu, Mufan, et al.
Published: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
by: Qian, Yulei, et al.
Published: (2024)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
by: Li, Bo, et al.
Published: (2026)

How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers
by: Lu, Xin, et al.
Published: (2024)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
by: Bohne, Jason, et al.
Published: (2025)

Mitigating the Bias of Large Language Model Evaluation
by: Zhou, Hongli, et al.
Published: (2024)

LLM-based Translation Inference with Iterative Bilingual Understanding
by: Chen, Andong, et al.
Published: (2024)

A Survey on Human Preference Learning for Large Language Models
by: Jiang, Ruili, et al.
Published: (2024)

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
by: ai, YuanLab., et al.
Published: (2026)

Faster MoE LLM Inference for Extremely Large Models
by: Yang, Haoqi, et al.
Published: (2025)

Beyond Token-Level Policy Gradients for Complex Reasoning with Large Language Models
by: Xu, Mufan, et al.
Published: (2026)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
by: Manzoni, Andrea
Published: (2026)

DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities
by: Lu, Xiangyu, et al.
Published: (2025)

MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training
by: Huang, Hui, et al.
Published: (2025)

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
by: Zhang, Shaoqing, et al.
Published: (2024)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
by: Deng, Jianing, et al.
Published: (2026)

Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation
by: Chen, Andong, et al.
Published: (2024)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
by: Zhu, Fengqi, et al.
Published: (2025)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
by: Tang, Yehui, et al.
Published: (2025)

ShardMemo: Masked MoE Routing for Sharded Agentic LLM Memory
by: Zhao, Yang, et al.
Published: (2026)

Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases
by: Gao, Bufan, et al.
Published: (2025)

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
by: Sun, Xingwu, et al.
Published: (2024)

AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
by: Zhang, Bo-Wen, et al.
Published: (2024)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
by: Huang, Quzhe, et al.
Published: (2024)

Long-form RewardBench: Evaluating Reward Models for Long-form Generation
by: Huang, Hui, et al.
Published: (2026)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
by: Tang, Yehui, et al.
Published: (2025)

PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning
by: Jung, Min Jae, et al.
Published: (2024)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
by: Shi, Jingze, et al.
Published: (2026)

UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE
by: Liu, Zhenyu, et al.
Published: (2025)

Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis
by: Chen, Andong, et al.
Published: (2025)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
by: Li, Shuhuai, et al.
Published: (2026)

MESA: Improving MoE Safety Alignment via Decentralized Expertise
by: Sun, Yitong, et al.
Published: (2026)