DINO-Tok: Adapting DINO for Visual Tokenizers
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908907917541376 |
|---|---|
| author | Jia, Mingkai Li, Mingxiao Shu, Zhijian Zheng, Anlin Fan, Liaoyuan Guo, Jiaxin Shi, Tianxing Lu, Dongyue Li, Zeming Guo, Xiaoyang Qi, Xiaojuan Long, Xiao-Xiao Zhang, Qian Tan, Ping Yin, Wei |
| author_facet | Jia, Mingkai Li, Mingxiao Shu, Zhijian Zheng, Anlin Fan, Liaoyuan Guo, Jiaxin Shi, Tianxing Lu, Dongyue Li, Zeming Guo, Xiaoyang Qi, Xiaojuan Long, Xiao-Xiao Zhang, Qian Tan, Ping Yin, Wei |
| contents | Recent advances in visual generation have emphasized the importance of Latent Generative Models (LGMs), which critically depend on effective visual tokenizers to bridge pixels and semantic representations. However, tokenizers constructed on pre-trained vision foundation models (VFMs) often struggle to balance semantic richness and reconstruction fidelity in high-dimensional latent spaces. In this paper, we introduce DINO-Tok, a visual tokenizer built upon a frozen DINO encoder that supports both continuous autoencoding (DINO-Tok-AE) and discrete vector-quantization (DINO-Tok-VQ). By unifying hierarchical representations from both shallow fine-grained features and deep global semantics into an information-complete latent space, DINO-Tok preserves texture details while maintaining \textit{semantic consistency} for generation. We further investigate VQ in frozen semantic feature spaces of high dimensionality, where information dilution and codebook collapse frequently arise. To address this issue, we propose Dominant-Subspace Quantization (DSQ), which leverages a global PCA analysis to select principal components while suppressing noisy dimensions, thereby stabilizing codebook optimization and improving reconstruction and generation quality. On ImageNet 256x256, DINO-Tok achieves strong reconstruction performance, achieving 0.28 rFID for continuous autoencoding and 1.10 rFID for discrete VQ, as well as strong few-step generation performance 1.82 gFID for diffusion and 2.44 gFID for autoregressive generation. These results demonstrate that pre-trained VFMs such as DINO can be directly adapted into high-fidelity, semantically aligned visual tokenizers for next-generation latent generative models. Code will be publicly available at https://github.com/MKJia/DINO-Tok. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20565 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DINO-Tok: Adapting DINO for Visual Tokenizers Jia, Mingkai Li, Mingxiao Shu, Zhijian Zheng, Anlin Fan, Liaoyuan Guo, Jiaxin Shi, Tianxing Lu, Dongyue Li, Zeming Guo, Xiaoyang Qi, Xiaojuan Long, Xiao-Xiao Zhang, Qian Tan, Ping Yin, Wei Computer Vision and Pattern Recognition Recent advances in visual generation have emphasized the importance of Latent Generative Models (LGMs), which critically depend on effective visual tokenizers to bridge pixels and semantic representations. However, tokenizers constructed on pre-trained vision foundation models (VFMs) often struggle to balance semantic richness and reconstruction fidelity in high-dimensional latent spaces. In this paper, we introduce DINO-Tok, a visual tokenizer built upon a frozen DINO encoder that supports both continuous autoencoding (DINO-Tok-AE) and discrete vector-quantization (DINO-Tok-VQ). By unifying hierarchical representations from both shallow fine-grained features and deep global semantics into an information-complete latent space, DINO-Tok preserves texture details while maintaining \textit{semantic consistency} for generation. We further investigate VQ in frozen semantic feature spaces of high dimensionality, where information dilution and codebook collapse frequently arise. To address this issue, we propose Dominant-Subspace Quantization (DSQ), which leverages a global PCA analysis to select principal components while suppressing noisy dimensions, thereby stabilizing codebook optimization and improving reconstruction and generation quality. On ImageNet 256x256, DINO-Tok achieves strong reconstruction performance, achieving 0.28 rFID for continuous autoencoding and 1.10 rFID for discrete VQ, as well as strong few-step generation performance 1.82 gFID for diffusion and 2.44 gFID for autoregressive generation. These results demonstrate that pre-trained VFMs such as DINO can be directly adapted into high-fidelity, semantically aligned visual tokenizers for next-generation latent generative models. Code will be publicly available at https://github.com/MKJia/DINO-Tok. |
| title | DINO-Tok: Adapting DINO for Visual Tokenizers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.20565 |