Copertina

SALMONN-omni: A Standalone Speech LLM without Codec Injection for Full-duplex Conversation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yu, Wenyi, Wang, Siyin, Yang, Xiaoyu, Chen, Xianzhao, Tian, Xiaohai, Zhang, Jun, Sun, Guangzhi, Lu, Lu, Wang, Yuxuan, Zhang, Chao
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
di: Yu, Wenyi, et al.
Pubblicazione: (2024)

video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
di: Sun, Guangzhi, et al.
Pubblicazione: (2024)

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
di: Wang, Siyin, et al.
Pubblicazione: (2025)

End-to-end Listen, Look, Speak and Act
di: Wang, Siyin, et al.
Pubblicazione: (2025)

SALMONN: Towards Generic Hearing Abilities for Large Language Models
di: Tang, Changli, et al.
Pubblicazione: (2023)

Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
di: Wang, Siyin, et al.
Pubblicazione: (2024)

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard
di: Yang, Yudong, et al.
Pubblicazione: (2025)

video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLM
di: Sun, Guangzhi, et al.
Pubblicazione: (2025)

Can Large Language Models Understand Spatial Audio?
di: Tang, Changli, et al.
Pubblicazione: (2024)

Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
di: Ao, Junyi, et al.
Pubblicazione: (2025)

video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models
di: Tang, Changli, et al.
Pubblicazione: (2025)

video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
di: Sun, Guangzhi, et al.
Pubblicazione: (2025)

A Full-duplex Speech Dialogue Scheme Based On Large Language Models
di: Wang, Peng, et al.
Pubblicazione: (2024)

FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction
di: Ge, Yuan, et al.
Pubblicazione: (2025)

Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement
di: Yang, Yudong, et al.
Pubblicazione: (2024)

Towards General Auditory Intelligence: Large Multimodal Models for Machine Listening and Speaking
di: Wang, Siyin, et al.
Pubblicazione: (2025)

Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
di: Wang, Siyin, et al.
Pubblicazione: (2024)

Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation
di: Zhao, Qiuming, et al.
Pubblicazione: (2025)

PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
di: Shi, Jiatong, et al.
Pubblicazione: (2025)

SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
di: Ao, Junyi, et al.
Pubblicazione: (2024)

Augmenting Open-Vocabulary Dysarthric Speech Assessment with Human Perceptual Supervision
di: Jia, Kaimeng, et al.
Pubblicazione: (2025)

VoCodec: An Efficient Lightweight Low-Bitrate Speech Codec
di: Yang, Leyan, et al.
Pubblicazione: (2026)

Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM
di: Shi, Jiatong, et al.
Pubblicazione: (2025)

Anatomy of the Modality Gap: Dissecting the Internal States of End-to-End Speech LLMs
di: Hsu, Ming-Hao, et al.
Pubblicazione: (2026)

Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization
di: Hu, Yuchen, et al.
Pubblicazione: (2024)

MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning
di: Zheng, Haolong, et al.
Pubblicazione: (2026)

Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
di: Wang, Junyi, et al.
Pubblicazione: (2026)

FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Rates
di: Li, Jiaqi, et al.
Pubblicazione: (2025)

AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluation
di: Wang, Lu, et al.
Pubblicazione: (2025)

SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
di: Qiang, Chunyu, et al.
Pubblicazione: (2025)

Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents
di: Sun, Guangzhi, et al.
Pubblicazione: (2024)

ClariCodec: Optimising Neural Speech Codes for 200bps Communication using Reinforcement Learning
di: Wang, Junyi, et al.
Pubblicazione: (2026)

AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
di: Shi, Jiacheng, et al.
Pubblicazione: (2026)

DisCo-Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech Codec
di: Li, Tao, et al.
Pubblicazione: (2025)

Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations
di: Jiang, Xue, et al.
Pubblicazione: (2025)

Cross-Utterance Conditioned VAE for Speech Generation
di: Li, Yang, et al.
Pubblicazione: (2023)

EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
di: Yao, Yiqun, et al.
Pubblicazione: (2025)

Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis
di: Yang, Yifan, et al.
Pubblicazione: (2025)

Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
di: Ji, Shengpeng, et al.
Pubblicazione: (2024)

CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing
di: Yue, Xianghu, et al.
Pubblicazione: (2024)