Cover Image

VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Long, Zuwei, Shen, Yunhang, Fu, Chaoyou, Gao, Heting, Li, Lijiang, Chen, Peixian, Zhang, Mengdan, Shao, Hang, Li, Jian, Peng, Jinlong, Cao, Haoyu, Li, Ke, Ji, Rongrong, Sun, Xing
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
by: Fu, Chaoyou, et al.
Published: (2025)

VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models
by: Zhou, Chenyu, et al.
Published: (2024)

DeepOmni: Towards Seamless and Smart Speech Interaction with Adaptive Modality-Specific MoE
by: Shao, Hang, et al.
Published: (2025)

Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
by: Li, Lijiang, et al.
Published: (2026)

Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy
by: Shen, Yunhang, et al.
Published: (2025)

Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
by: Li, Xudong, et al.
Published: (2025)

VITA: Towards Open-Source Interactive Omni Multimodal LLM
by: Fu, Chaoyou, et al.
Published: (2024)

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
by: Gao, Timin, et al.
Published: (2024)

LUCY: Linguistic Understanding and Control Yielding Early Stage of Her
by: Gao, Heting, et al.
Published: (2025)

MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
by: Fu, Chaoyou, et al.
Published: (2023)

Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM
by: Wang, Xiong, et al.
Published: (2024)

VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing
by: Xu, Jiacheng, et al.
Published: (2026)

VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing, Speaking, and Acting
by: Liu, Xiaoyu, et al.
Published: (2025)

VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation
by: Dong, Shaoqi, et al.
Published: (2025)

Streaming Video Instruction Tuning
by: Xia, Jiaer, et al.
Published: (2025)

AlignVSR: Audio-Visual Cross-Modal Alignment for Visual Speech Recognition
by: Liu, Zehua, et al.
Published: (2024)

Interleaved-Modal Chain-of-Thought
by: Gao, Jun, et al.
Published: (2024)

Towards Universal Modal Tracking with Online Dense Temporal Token Learning
by: Zheng, Yaozong, et al.
Published: (2025)

Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?
by: Jiang, Hongbo, et al.
Published: (2026)

M4-BLIP: Advancing Multi-Modal Media Manipulation Detection through Face-Enhanced Local Analysis
by: Wu, Hang, et al.
Published: (2025)

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
by: Sun, Zhen, et al.
Published: (2025)

Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model
by: Xue, Jinlong, et al.
Published: (2024)

MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples
by: Chen, Tao, et al.
Published: (2023)

Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models
by: Li, Xudong, et al.
Published: (2024)

Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
by: Fu, Chaoyou, et al.
Published: (2024)

A General and Efficient Training for Transformer via Token Expansion
by: Huang, Wenxuan, et al.
Published: (2024)

LongCat-Audio-Codec: An Audio Tokenizer and Detokenizer Solution Designed for Speech Large Language Models
by: Zhao, Xiaohan, et al.
Published: (2025)

MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models
by: Gong, Yitian, et al.
Published: (2026)

Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inference
by: Zhan, Wengyi, et al.
Published: (2025)

Cross-Lingual Interleaving for Speech Language Models
by: Moumen, Adel, et al.
Published: (2025)

Woodpecker: Hallucination Correction for Multimodal Large Language Models
by: Yin, Shukang, et al.
Published: (2023)

Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation
by: Wei, Kun, et al.
Published: (2023)

FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compression
by: Tong, Bo, et al.
Published: (2024)

Multi-Modal Prompt Learning on Blind Image Quality Assessment
by: Pan, Wensheng, et al.
Published: (2024)

Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction
by: Guichoux, Téo, et al.
Published: (2025)

Separate in the Speech Chain: Cross-Modal Conditional Audio-Visual Target Speech Extraction
by: Mu, Zhaoxi, et al.
Published: (2024)

Retrieval Augmented Generation in Prompt-based Text-to-Speech Synthesis with Context-Aware Contrastive Language-Audio Pretraining
by: Xue, Jinlong, et al.
Published: (2024)

Audio-Visual Cross-Modal Compression for Generative Face Video Coding
by: Xu, Youmin, et al.
Published: (2025)

VITA: Vision-to-Action Flow Matching Policy
by: Gao, Dechen, et al.
Published: (2025)

Scaling Open Discrete Audio Foundation Models with Interleaved Semantic, Acoustic, and Text Tokens
by: Manakul, Potsawee, et al.
Published: (2026)