Archon: A Unified Multimodal Model for Holistic Digital Human Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918530094465024 |
|---|---|
| author | Bao, Chong Liu, Shichen Yu, Lijun Futschik, David Moschoglou, Stylianos Srivastava, Shefali Bai, Ziqian Tan, Feitong Zhang, Guofeng Cui, Zhaopeng Fanello, Sean Zhang, Yinda |
| author_facet | Bao, Chong Liu, Shichen Yu, Lijun Futschik, David Moschoglou, Stylianos Srivastava, Shefali Bai, Ziqian Tan, Feitong Zhang, Guofeng Cui, Zhaopeng Fanello, Sean Zhang, Yinda |
| contents | Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visual content, remains an open challenge. In this paper, we present Archon, a fully pretrained, human-centric unified multimodal model for holistic avatar generation. Archon unifies seven modalities with modality-specific tokenizers, and a native autoregressive unified multimodal model pretrained on synchronized modalities and 72 diverse tasks to model holistic joint distributions. To address the token explosion challenge in high-fidelity talking videos, we introduce a memory-efficient semantic video reparameterization, achieving 4x token reduction while preserving fine-grained dynamics, coupled with a semantic-driven video diffusion decoder. We further propose a "Thinking in Modality" that decomposes ambiguous cross-modal tasks into stepwise thinking in an alternative chain of modality, progressively enhancing fidelity and controllability. Extensive experiments demonstrate that Archon achieves superior or comparable performance across diverse digital human generation tasks, validating the effectiveness of our unified framework. Project page: https://zju3dv.github.io/archon/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30311 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Archon: A Unified Multimodal Model for Holistic Digital Human Generation Bao, Chong Liu, Shichen Yu, Lijun Futschik, David Moschoglou, Stylianos Srivastava, Shefali Bai, Ziqian Tan, Feitong Zhang, Guofeng Cui, Zhaopeng Fanello, Sean Zhang, Yinda Computer Vision and Pattern Recognition Artificial Intelligence Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visual content, remains an open challenge. In this paper, we present Archon, a fully pretrained, human-centric unified multimodal model for holistic avatar generation. Archon unifies seven modalities with modality-specific tokenizers, and a native autoregressive unified multimodal model pretrained on synchronized modalities and 72 diverse tasks to model holistic joint distributions. To address the token explosion challenge in high-fidelity talking videos, we introduce a memory-efficient semantic video reparameterization, achieving 4x token reduction while preserving fine-grained dynamics, coupled with a semantic-driven video diffusion decoder. We further propose a "Thinking in Modality" that decomposes ambiguous cross-modal tasks into stepwise thinking in an alternative chain of modality, progressively enhancing fidelity and controllability. Extensive experiments demonstrate that Archon achieves superior or comparable performance across diverse digital human generation tasks, validating the effectiveness of our unified framework. Project page: https://zju3dv.github.io/archon/. |
| title | Archon: A Unified Multimodal Model for Holistic Digital Human Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2605.30311 |