Copertina

From Visuals to Vocabulary: Establishing Equivalence Between Image and Text Token Through Autoregressive Pre-training in MLLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Mingxiao, Qu, Fang, Chen, Zhanpeng, Su, Na, Zhong, Zhizhou, Chen, Ziyang, Du, Nan, Li, Xiaolong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computer Vision and Pattern Recognition
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

VISTA: Enhancing Vision-Text Alignment in MLLMs via Cross-Modal Mutual Information Maximization
di: Li, Mingxiao, et al.
Pubblicazione: (2025)

Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding
di: Chen, Zhanpeng, et al.
Pubblicazione: (2025)

RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents
di: Zhang, Zijing, et al.
Pubblicazione: (2025)

Time Matters: Enhancing Pre-trained News Recommendation Models with Robust User Dwell Time Injection
di: Jiang, Hao, et al.
Pubblicazione: (2024)

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models
di: Qu, Tingyu, et al.
Pubblicazione: (2024)

Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation
di: Chen, Ruoyu, et al.
Pubblicazione: (2025)

Visual Semantic Description Generation with MLLMs for Image-Text Matching
di: Chen, Junyu, et al.
Pubblicazione: (2025)

Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation
di: Li, Yongqi, et al.
Pubblicazione: (2024)

V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation
di: Zhang, Guiwei, et al.
Pubblicazione: (2025)

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
di: Zhong, Zexuan, et al.
Pubblicazione: (2024)

Text Embeddings by Weakly-Supervised Contrastive Pre-training
di: Wang, Liang, et al.
Pubblicazione: (2022)

Generative Pre-trained Autoregressive Diffusion Transformer
di: Zhang, Yuan, et al.
Pubblicazione: (2025)

Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence
di: Işık, İlker, et al.
Pubblicazione: (2024)

EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs
di: Chen, Yuhao, et al.
Pubblicazione: (2026)

TokenCompose: Text-to-Image Diffusion with Token-level Supervision
di: Wang, Zirui, et al.
Pubblicazione: (2023)

Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens
di: Fan, Lijie, et al.
Pubblicazione: (2024)

Unified Autoregressive Visual Generation and Understanding with Continuous Tokens
di: Fan, Lijie, et al.
Pubblicazione: (2025)

Towards Scalable Pre-training of Visual Tokenizers for Generation
di: Yao, Jingfeng, et al.
Pubblicazione: (2025)

Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval
di: Su, Weihang, et al.
Pubblicazione: (2023)

DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies
di: Song, Wei, et al.
Pubblicazione: (2025)

Growing Visual Generative Capacity for Pre-Trained MLLMs
di: Wang, Hanyu, et al.
Pubblicazione: (2025)

On the Equivalence Between Auto-Regressive Next Token Prediction and Full-Item-Vocabulary Maximum Likelihood Estimation in Generative Recommendation--A Short Note
di: Huang, Yusheng, et al.
Pubblicazione: (2026)

GridPrune: From "Where to Look" to "What to Select" in Visual Token Pruning for MLLMs
di: Duan, Yuxiang, et al.
Pubblicazione: (2025)

Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training
di: Liu, Yi, et al.
Pubblicazione: (2025)

QG-VTC: Question-Guided Visual Token Compression in MLLMs for Efficient VQA
di: Li, Shuai, et al.
Pubblicazione: (2025)

On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
di: Ke, Yekun, et al.
Pubblicazione: (2025)

TokenMark: A Modality-Agnostic Watermark for Pre-trained Transformers
di: Xu, Hengyuan, et al.
Pubblicazione: (2024)

InsightTok: Improving Text and Face Fidelity in Discrete Tokenization for Autoregressive Image Generation
di: Yue, Yang, et al.
Pubblicazione: (2026)

Scaling LLM Pre-training with Vocabulary Curriculum
di: Yu, Fangyuan
Pubblicazione: (2025)

Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning
di: Wang, Bohan, et al.
Pubblicazione: (2025)

DataVisT5: A Pre-trained Language Model for Jointly Understanding Text and Data Visualization
di: Wan, Zhuoyue, et al.
Pubblicazione: (2024)

Multimodal Autoregressive Pre-training of Large Vision Encoders
di: Fini, Enrico, et al.
Pubblicazione: (2024)

InfoTok: Information-Theoretic Regularization for Capacity-Constrained Shared Visual Tokenization in Unified MLLMs
di: Tang, Lv, et al.
Pubblicazione: (2026)

MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarization
di: Balde, Gunjan, et al.
Pubblicazione: (2024)

Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
di: Su, Yongyi, et al.
Pubblicazione: (2025)

Autoregressive Pre-Training on Pixels and Texts
di: Chai, Yekun, et al.
Pubblicazione: (2024)

On the Effect of Token Merging on Pre-trained Models for Code
di: Saad, Mootez, et al.
Pubblicazione: (2025)

Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-training
di: Su, Tongkun, et al.
Pubblicazione: (2024)

Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image
di: Jiao, Pengkun, et al.
Pubblicazione: (2024)

The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Loss
di: Li, Bozhou, et al.
Pubblicazione: (2025)