L2P: Unlocking Latent Potential for Pixel Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Zhennan, Zhu, Junwei, Chen, Xu, Zhang, Jiangning, Chen, Jiawei, Zeng, Zhuoqi, Zhang, Wei, Wang, Chengjie, Yang, Jian, Tai, Ying
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866909035940282368
author	Chen, Zhennan Zhu, Junwei Chen, Xu Zhang, Jiangning Chen, Jiawei Zeng, Zhuoqi Zhang, Wei Wang, Chengjie Yang, Jian Tai, Ying
author_facet	Chen, Zhennan Zhu, Junwei Chen, Xu Zhang, Jiangning Chen, Jiawei Zeng, Zhuoqi Zhang, Wei Wang, Chengjie Yang, Jian Tai, Ying
contents	Pixel diffusion models have recently regained attention for visual generation. However, training advanced pixel-space models from scratch demands prohibitive computational and data resources. To address this, we propose the Latent-to-Pixel (L2P) transfer paradigm, an efficient framework that directly harnesses the rich knowledge of pre-trained LDMs to build powerful pixel-space models. Specifically, L2P discards the VAE in favor of large-patch tokenization and freezes the source LDM's intermediate layers, exclusively training shallow layers to learn the latent-to-pixel transformation. By utilizing LDM-generated synthetic images as the sole training corpus, L2P fits an already smooth data manifold, enabling rapid convergence with zero real-data collection. This strategy allows L2P to seamlessly migrate massive latent priors to the pixel space using only 8 GPUs. Furthermore, eliminating the VAE memory bottleneck unlocks native 4K ultra-high resolution generation. Extensive experiments across mainstream LDM architectures show that L2P incurs negligible training overhead, yet performs on par with the source LDM on DPG-Bench and reaches 93% performance on GenEval.
format	Preprint
id	arxiv_https___arxiv_org_abs_2605_12013
institution	arXiv
publishDate	2026
record_format	arxiv
spellingShingle	L2P: Unlocking Latent Potential for Pixel Generation Chen, Zhennan Zhu, Junwei Chen, Xu Zhang, Jiangning Chen, Jiawei Zeng, Zhuoqi Zhang, Wei Wang, Chengjie Yang, Jian Tai, Ying Computer Vision and Pattern Recognition Artificial Intelligence Pixel diffusion models have recently regained attention for visual generation. However, training advanced pixel-space models from scratch demands prohibitive computational and data resources. To address this, we propose the Latent-to-Pixel (L2P) transfer paradigm, an efficient framework that directly harnesses the rich knowledge of pre-trained LDMs to build powerful pixel-space models. Specifically, L2P discards the VAE in favor of large-patch tokenization and freezes the source LDM's intermediate layers, exclusively training shallow layers to learn the latent-to-pixel transformation. By utilizing LDM-generated synthetic images as the sole training corpus, L2P fits an already smooth data manifold, enabling rapid convergence with zero real-data collection. This strategy allows L2P to seamlessly migrate massive latent priors to the pixel space using only 8 GPUs. Furthermore, eliminating the VAE memory bottleneck unlocks native 4K ultra-high resolution generation. Extensive experiments across mainstream LDM architectures show that L2P incurs negligible training overhead, yet performs on par with the source LDM on DPG-Bench and reaches 93% performance on GenEval.
title	L2P: Unlocking Latent Potential for Pixel Generation
topic	Computer Vision and Pattern Recognition Artificial Intelligence
url	https://arxiv.org/abs/2605.12013