Cover Image

Generalizability of Mixture of Domain-Specific Adapters from the Lens of Signed Weight Directions and its Application to Effective Model Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nguyen, Tuc, Le, Thai
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adapters Mixup: Mixing Parameter-Efficient Adapters to Enhance the Adversarial Robustness of Fine-tuned Pre-trained Text Classifiers
by: Nguyen, Tuc, et al.
Published: (2024)

ATLAS: Adaptive Test-Time Latent Steering with External Verifiers for Enhancing LLMs Reasoning
by: Nguyen, Tuc, et al.
Published: (2026)

Unraveling Interwoven Roles of Large Language Models in Authorship Privacy: Obfuscation, Mimicking, and Verification
by: Nguyen, Tuc, et al.
Published: (2025)

NoMatterXAI: Generating "No Matter What" Alterfactual Examples for Explaining Black-Box Text Classification Models
by: Nguyen, Tuc, et al.
Published: (2024)

ShareChat: A Dataset of Chatbot Conversations in the Wild
by: Yan, Yueru, et al.
Published: (2025)

Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
by: Dong, Zican, et al.
Published: (2025)

Fast and Effective Weight Update for Pruned Large Language Models
by: Boža, Vladimír
Published: (2024)

Topological Data Analysis Applications in Natural Language Processing: A Survey
by: Uchendu, Adaku, et al.
Published: (2024)

All-in-One Tuning and Structural Pruning for Domain-Specific LLMs
by: Lu, Lei, et al.
Published: (2024)

EditLens: Quantifying the Extent of AI Editing in Text
by: Thai, Katherine, et al.
Published: (2025)

MoKA: Mixture of Kronecker Adapters
by: Sadeghi, Mohammadreza, et al.
Published: (2025)

Flexible and Effective Mixing of Large Language Models into a Mixture of Domain Experts
by: Lee, Rhui Dih, et al.
Published: (2024)

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models
by: Guo, Hongcheng, et al.
Published: (2025)

Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
by: Jiang, Songtao, et al.
Published: (2024)

Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
by: Siam, M. K. Khalidi, et al.
Published: (2026)

DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models
by: Fu, Jiachen, et al.
Published: (2025)

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs
by: Fu, Yao, et al.
Published: (2025)

MoDEM: Mixture of Domain Expert Models
by: Simonds, Toby, et al.
Published: (2024)

FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and Finetuning
by: Bhattacharyya, Chaitali, et al.
Published: (2025)

signwriting-evaluation: Effective Sign Language Evaluation via SignWriting
by: Moryossef, Amit, et al.
Published: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

Breaking Down Bias: On The Limits of Generalizable Pruning Strategies
by: Ma, Sibo, et al.
Published: (2025)

Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models
by: Li, Chengzhengxu, et al.
Published: (2024)

Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration
by: Wu, Guangxin, et al.
Published: (2026)

MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
by: Cao, Jie, et al.
Published: (2025)

RAID: Refusal-Aware and Integrated Decoding for Jailbreaking LLMs
by: Nguyen, Tuan T., et al.
Published: (2025)

Towards Domain Specification of Embedding Models in Medicine
by: Khodadad, Mohammad, et al.
Published: (2025)

Pruning as a Domain-specific LLM Extractor
by: Zhang, Nan, et al.
Published: (2024)

The Impact of LoRA Adapters on LLMs for Clinical Text Classification Under Computational and Data Constraints
by: Le, Thanh-Dung, et al.
Published: (2024)

Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
by: Chen, Hanting, et al.
Published: (2025)

MSA-ASR: Efficient Multilingual Speaker Attribution with frozen ASR Models
by: Nguyen, Thai-Binh, et al.
Published: (2024)

MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts
by: Liao, Yusheng, et al.
Published: (2024)

READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling
by: Nguyen, Thong, et al.
Published: (2023)

DMoERM: Recipes of Mixture-of-Experts for Effective Reward Modeling
by: Quan, Shanghaoran
Published: (2024)

GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
by: Tang, Yixuan, et al.
Published: (2025)

Hybrid Training Approaches for LLMs: Leveraging Real and Synthetic Data to Enhance Model Performance in Domain-Specific Applications
by: Zhezherau, Alexey, et al.
Published: (2024)

On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions
by: Wang, Weiqi, et al.
Published: (2024)

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
by: Bai, Sikai, et al.
Published: (2025)

Lightweight Zero-shot Text-to-Speech with Mixture of Adapters
by: Fujita, Kenichi, et al.
Published: (2024)

CLIP-Adapter: Better Vision-Language Models with Feature Adapters
by: Gao, Peng, et al.
Published: (2021)