Cover Image

Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Onda, Kentaro, Futami, Hayato, Kashiwagi, Yosuke, Tsunoo, Emiru, Watanabe, Shinji
Format:	Preprint
Published:	2026
Subjects:	Sound
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Differentiable K-means for Fully-optimized Discrete Token-based ASR
by: Onda, Kentaro, et al.
Published: (2025)

Hypothesis Clustering and Merging: Novel MultiTalker Speech Recognition with Speaker Tokens
by: Kashiwagi, Yosuke, et al.
Published: (2024)

Decoder-only Architecture for Speech Recognition with CTC Prompts and Text Data Augmentation
by: Tsunoo, Emiru, et al.
Published: (2023)

Spiralformer: Low Latency Encoder for Streaming Speech Recognition with Circular Layer Skipping and Early Exiting
by: Tsunoo, Emiru, et al.
Published: (2025)

Rapid Language Adaptation for Multilingual E2E Speech Recognition Using Encoder Prompting
by: Kashiwagi, Yosuke, et al.
Published: (2024)

Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback
by: Arora, Siddhant, et al.
Published: (2026)

Chain-of-Thought Reasoning in Streaming Full-Duplex End-to-End Spoken Dialogue Systems
by: Arora, Siddhant, et al.
Published: (2025)

Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs
by: Futami, Hayato, et al.
Published: (2025)

Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model
by: Futami, Hayato, et al.
Published: (2024)

Chain-of-Thought Training for Open E2E Spoken Dialogue Systems
by: Arora, Siddhant, et al.
Published: (2025)

Decoder-only Architecture for Streaming End-to-end Speech Recognition
by: Tsunoo, Emiru, et al.
Published: (2024)

UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions
by: Arora, Siddhant, et al.
Published: (2023)

Whale: Large-Scale multilingual ASR model with w2v-BERT and E-Branchformer with large speech data
by: Kashiwagi, Yosuke, et al.
Published: (2025)

Benchmarking Prosody Encoding in Discrete Speech Tokens
by: Onda, Kentaro, et al.
Published: (2025)

ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems
by: Arora, Siddhant, et al.
Published: (2025)

Advanced Modeling of Interlanguage Speech Intelligibility Benefit with L1-L2 Multi-Task Learning Using Differentiable K-Means for Accent-Robust Discrete Token-Based ASR
by: Onda, Kentaro, et al.
Published: (2026)

Task Arithmetic for Language Expansion in Speech Translation
by: Cheng, Yao-Fei, et al.
Published: (2024)

Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens
by: Ulgen, Ismail Rasim, et al.
Published: (2025)

Causal Speech Enhancement with Predicting Semantics based on Quantized Self-supervised Learning Features
by: Tsunoo, Emiru, et al.
Published: (2024)

Prosodically Enhanced Foreign Accent Simulation by Discrete Token-based Resynthesis Only with Native Speech Corpora
by: Onda, Kentaro, et al.
Published: (2025)

Discrete Tokens Exhibit Interlanguage Speech Intelligibility Benefit: an Analytical Study Towards Accent-robust ASR Only with Native Speech Data
by: Onda, Kentaro, et al.
Published: (2025)

PAST: Phonetic-Acoustic Speech Tokenizer
by: Har-Tuv, Nadav, et al.
Published: (2025)

DialogueSidon: Recovering Full-Duplex Dialogue Tracks from In-the-Wild Dialogue Audio
by: Nakata, Wataru, et al.
Published: (2026)

LibriTTS-VI: A Public Corpus and Novel Methods for Efficient Voice Impression Control
by: Ohmura, Junki, et al.
Published: (2025)

SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition
by: Wang, Pu, et al.
Published: (2026)

Segment-Level Vectorized Beam Search Based on Partially Autoregressive Inference
by: Someki, Masao, et al.
Published: (2023)

EmotionThinker: Prosody-Aware Reinforcement Learning for Explainable Speech Emotion Reasoning
by: Wang, Dingdong, et al.
Published: (2026)

CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR
by: Shakeel, Muhammad, et al.
Published: (2026)

Improving Multilingual Speech Models on ML-SUPERB 2.0: Fine-tuning with Data Augmentation and LID-Aware CTC
by: Wang, Qingzheng, et al.
Published: (2025)

LALM-as-a-Judge: Benchmarking Large Audio-Language Models for Safety Evaluation in Multi-Turn Spoken Dialogues
by: Ivry, Amir, et al.
Published: (2026)

A Pilot Study of GSLM-based Simulation of Foreign Accentuation Only Using Native Speech Corpora
by: Onda, Kentaro, et al.
Published: (2024)

Sign-to-Speech Prosody Transfer via Sign Reconstruction-based GAN
by: Manabe, Toranosuke, et al.
Published: (2026)

DuoTok: Source-Aware Dual-Track Tokenization for Multi-Track Music Language Modeling
by: Lin, Rui, et al.
Published: (2025)

Geolocation-Aware Robust Spoken Language Identification
by: Wang, Qingzheng, et al.
Published: (2025)

Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization
by: Fang, Zheng, et al.
Published: (2026)

CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese
by: Chen, Dazhong, et al.
Published: (2025)

Multi-Class-Token Transformer for Multitask Self-supervised Music Information Retrieval
by: Kong, Yuexuan, et al.
Published: (2025)

VocalNet-M2: Advancing Low-Latency Spoken Language Modeling via Integrated Multi-Codebook Tokenization and Multi-Token Prediction
by: Wang, Yuhao, et al.
Published: (2025)

Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis
by: Li, Jialu, et al.
Published: (2023)

Why Your Tokenizer Fails in Information Fusion: A Timing-Aware Pre-Quantization Fusion for Video-Enhanced Audio Tokenization
by: Zhang, Xiangyu, et al.
Published: (2026)