Image de couverture de livre

SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yu, Wenyi, Wang, Siyin, Yang, Xiaoyu, Chen, Xianzhao, Tian, Xiaohai, Zhang, Jun, Sun, Guangzhi, Lu, Lu, Wang, Yuxuan, Zhang, Chao
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
par: Yu, Wenyi, et autres
Publié: (2024)

video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
par: Sun, Guangzhi, et autres
Publié: (2024)

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
par: Wang, Siyin, et autres
Publié: (2025)

End-to-end Listen, Look, Speak and Act
par: Wang, Siyin, et autres
Publié: (2025)

SALMONN: Towards Generic Hearing Abilities for Large Language Models
par: Tang, Changli, et autres
Publié: (2023)

Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
par: Wang, Siyin, et autres
Publié: (2024)

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard
par: Yang, Yudong, et autres
Publié: (2025)

video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLM
par: Sun, Guangzhi, et autres
Publié: (2025)

Can Large Language Models Understand Spatial Audio?
par: Tang, Changli, et autres
Publié: (2024)

Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
par: Ao, Junyi, et autres
Publié: (2025)

video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models
par: Tang, Changli, et autres
Publié: (2025)

video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
par: Sun, Guangzhi, et autres
Publié: (2025)

A Full-duplex Speech Dialogue Scheme Based On Large Language Models
par: Wang, Peng, et autres
Publié: (2024)

FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction
par: Ge, Yuan, et autres
Publié: (2025)

Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement
par: Yang, Yudong, et autres
Publié: (2024)

Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking
par: Wang, Siyin, et autres
Publié: (2025)

Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
par: Wang, Siyin, et autres
Publié: (2024)

Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation
par: Zhao, Qiuming, et autres
Publié: (2025)

PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
par: Shi, Jiatong, et autres
Publié: (2025)

SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
par: Ao, Junyi, et autres
Publié: (2024)

Augmenting Open-Vocabulary Dysarthric Speech Assessment with Human Perceptual Supervision
par: Jia, Kaimeng, et autres
Publié: (2025)

VoCodec: An Efficient Lightweight Low-Bitrate Speech Codec
par: Yang, Leyan, et autres
Publié: (2026)

Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM
par: Shi, Jiatong, et autres
Publié: (2025)

Anatomy of the Modality Gap: Dissecting the Internal States of End-to-End Speech LLMs
par: Hsu, Ming-Hao, et autres
Publié: (2026)

Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization
par: Hu, Yuchen, et autres
Publié: (2024)

MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning
par: Zheng, Haolong, et autres
Publié: (2026)

Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
par: Wang, Junyi, et autres
Publié: (2026)

FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Rates
par: Li, Jiaqi, et autres
Publié: (2025)

AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluation
par: Wang, Lu, et autres
Publié: (2025)

SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
par: Qiang, Chunyu, et autres
Publié: (2025)

Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents
par: Sun, Guangzhi, et autres
Publié: (2024)

ClariCodec: Optimising Neural Speech Codes for 200bps Communication using Reinforcement Learning
par: Wang, Junyi, et autres
Publié: (2026)

AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
par: Shi, Jiacheng, et autres
Publié: (2026)

DisCo-Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech Codec
par: Li, Tao, et autres
Publié: (2025)

Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations
par: Jiang, Xue, et autres
Publié: (2025)

Cross-Utterance Conditioned VAE for Speech Generation
par: Li, Yang, et autres
Publié: (2023)

EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
par: Yao, Yiqun, et autres
Publié: (2025)

Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis
par: Yang, Yifan, et autres
Publié: (2025)

Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
par: Ji, Shengpeng, et autres
Publié: (2024)

CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing
par: Yue, Xianghu, et autres
Publié: (2024)