HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Yu, Ma, Lichen, Guo, Zipeng, Shan, Xinyuan, Fu, Jingling, Chen, Dong, Huang, Junshi, Li, Yan
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866916015786426368
author	He, Yu Ma, Lichen Guo, Zipeng Shan, Xinyuan Fu, Jingling Chen, Dong Huang, Junshi Li, Yan
author_facet	He, Yu Ma, Lichen Guo, Zipeng Shan, Xinyuan Fu, Jingling Chen, Dong Huang, Junshi Li, Yan
contents	Pixel-space diffusion models bypass the reconstruction bottleneck of Variational Autoencoders (VAEs) but face a fundamental "granularity dilemma": capturing global semantics favors large patch scales, while generating high-fidelity details demands fine-grained inputs. To address this issue, we propose HyperDiT, a unified framework establishing Hyper-Connected Cross-Scale Interactions to bridge the semantic and pixel manifold. Diverging from injecting semantics by AdaLN, HyperDiT utilizes Cross-Attention mechanisms, enabling fine-grained tokens to query multi-level semantic anchors globally. To resolve the spatial mismatch during multi-scale interactions, we introduce Scale-Aware Rotary Position Embedding (SA-RoPE) to ensure precise geometric alignment among tokens of varying patch sizes. Furthermore, we incorporate Registers to learn the dense semantics from a pretrained Visual Foundation Model (VFM), effectively reducing generation hallucination and artifacts. Extensive experiments demonstrate that HyperDiT achieves state-of-the-art (SoTA) FID of $\mathbf{1.56}$ on ImageNet $256\times256$ directly within the pixel space. By combining the fine-grained stream with semantic guidance, HyperDiT offers a superior paradigm for high-fidelity pixel generation.
format	Preprint
id	arxiv_https___arxiv_org_abs_2605_15741
institution	arXiv
publishDate	2026
record_format	arxiv
spellingShingle	HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion He, Yu Ma, Lichen Guo, Zipeng Shan, Xinyuan Fu, Jingling Chen, Dong Huang, Junshi Li, Yan Computer Vision and Pattern Recognition Pixel-space diffusion models bypass the reconstruction bottleneck of Variational Autoencoders (VAEs) but face a fundamental "granularity dilemma": capturing global semantics favors large patch scales, while generating high-fidelity details demands fine-grained inputs. To address this issue, we propose HyperDiT, a unified framework establishing Hyper-Connected Cross-Scale Interactions to bridge the semantic and pixel manifold. Diverging from injecting semantics by AdaLN, HyperDiT utilizes Cross-Attention mechanisms, enabling fine-grained tokens to query multi-level semantic anchors globally. To resolve the spatial mismatch during multi-scale interactions, we introduce Scale-Aware Rotary Position Embedding (SA-RoPE) to ensure precise geometric alignment among tokens of varying patch sizes. Furthermore, we incorporate Registers to learn the dense semantics from a pretrained Visual Foundation Model (VFM), effectively reducing generation hallucination and artifacts. Extensive experiments demonstrate that HyperDiT achieves state-of-the-art (SoTA) FID of $\mathbf{1.56}$ on ImageNet $256\times256$ directly within the pixel space. By combining the fine-grained stream with semantic guidance, HyperDiT offers a superior paradigm for high-fidelity pixel generation.
title	HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion
topic	Computer Vision and Pattern Recognition
url	https://arxiv.org/abs/2605.15741