MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917308954312704 |
|---|---|
| author | Kong, Lingshun Zhang, Jiawei Duan, Zhengpeng Wu, Xiaohe Yang, Yueqi Wang, Xiaotao Zou, Dongqing Lei, Lei Pan, Jinshan |
| author_facet | Kong, Lingshun Zhang, Jiawei Duan, Zhengpeng Wu, Xiaohe Yang, Yueqi Wang, Xiaotao Zou, Dongqing Lei, Lei Pan, Jinshan |
| contents | All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02710 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration Kong, Lingshun Zhang, Jiawei Duan, Zhengpeng Wu, Xiaohe Yang, Yueqi Wang, Xiaotao Zou, Dongqing Lei, Lei Pan, Jinshan Computer Vision and Pattern Recognition All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task. |
| title | MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.02710 |