Cover Image

Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and Language

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yeo, Jeong Hun, Kim, Chae Won, Kim, Hyunjun, Rha, Hyeongseop, Han, Seunghee, Cheng, Wen-Huang, Ro, Yong Man
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Computation and Language Audio and Speech Processing Image and Video Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Inclusive Communication: A Unified Framework for Generating Spoken Language from Sign, Lip, and Audio
by: Yeo, Jeong Hun, et al.
Published: (2025)

Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge
by: Kim, Minsu, et al.
Published: (2023)

Decoding Strategies for Diffusion-Based ASR: A Systematic Evaluation of Confidence-Based Thresholding
by: Yeo, Jeong Hun, et al.
Published: (2026)

Efficient Training for Multilingual Visual Speech Recognition: Pre-training with Discretized Visual Speech Representation
by: Kim, Minsu, et al.
Published: (2024)

MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens
by: Yeo, Jeong Hun, et al.
Published: (2025)

Where Visual Speech Meets Language: VSP-LLM Framework for Efficient and Context-Aware Visual Speech Processing
by: Yeo, Jeong Hun, et al.
Published: (2024)

Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations
by: Yeo, Jeong Hun, et al.
Published: (2025)

Exploring Phonetic Context-Aware Lip-Sync For Talking Face Generation
by: Park, Se Jin, et al.
Published: (2023)

Visual Speech Recognition for Languages with Limited Labeled Data using Automatic Labels from Whisper
by: Yeo, Jeong Hun, et al.
Published: (2023)

AKVSR: Audio Knowledge Empowered Visual Speech Recognition by Compressing Audio Knowledge of a Pretrained Model
by: Yeo, Jeong Hun, et al.
Published: (2023)

SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer
by: Park, Young-Hu, et al.
Published: (2025)

LipVoicer: Generating Speech from Silent Videos Guided by Lip Reading
by: Yemini, Yochai, et al.
Published: (2023)

Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation
by: Kim, Minsu, et al.
Published: (2023)

TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
by: Kim, Minsu, et al.
Published: (2024)

Beyond Lips: Integrating Gesture and Lip Cues for Robust Audio-visual Speaker Extraction
by: Pan, Zexu, et al.
Published: (2026)

LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Models
by: Richter, Julius, et al.
Published: (2025)

Fairness of Automatic Speech Recognition in Cleft Lip and Palate Speech
by: Bhattacharjee, Susmita, et al.
Published: (2025)

Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-Reading
by: Luo, Songtao, et al.
Published: (2023)

Towards Accurate Lip-to-Speech Synthesis in-the-Wild
by: Hegde, Sindhu, et al.
Published: (2024)

LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition
by: Ghosh, Sreyan, et al.
Published: (2024)

SLD-L2S: Hierarchical Subspace Latent Diffusion for High-Fidelity Lip to Speech Synthesis
by: Liang, Yifan, et al.
Published: (2026)

Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition
by: Kim, Minsu, et al.
Published: (2023)

Evaluating Automatic Speech Recognition Systems for Korean Meteorological Experts
by: Park, ChaeHun, et al.
Published: (2024)

Improving Lip-synchrony in Direct Audio-Visual Speech-to-Speech Translation
by: Goncalves, Lucas, et al.
Published: (2024)

Text-driven Talking Face Synthesis by Reprogramming Audio-driven Models
by: Choi, Jeongsoo, et al.
Published: (2023)

AdaptVC: High Quality Voice Conversion with Adaptive Learning
by: Kim, Jaehun, et al.
Published: (2025)

NaturalL2S: End-to-End High-quality Multispeaker Lip-to-Speech Synthesis with Differential Digital Signal Processing
by: Liang, Yifan, et al.
Published: (2025)

Long-Form Speech Generation with Spoken Language Models
by: Park, Se Jin, et al.
Published: (2024)

Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder
by: Dai, Yusheng, et al.
Published: (2023)

When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining
by: Yeo, Juan, et al.
Published: (2024)

Adapting a Text-to-Audio Model for Room Impulse Response Generation
by: Kim, Kirak, et al.
Published: (2026)

AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation
by: Choi, Jeongsoo, et al.
Published: (2023)

Voice Biomarker Analysis and Automated Severity Classification of Dysarthric Speech in a Multilingual Context
by: Yeo, Eunjung
Published: (2024)

Scaling Self-Supervised Speech Models Uncovers Deep Linguistic Relationships: Evidence from the Pacific Cluster
by: Kim, Minu, et al.
Published: (2026)

How Far Do SSL Speech Models Listen for Tone? Temporal Focus of Tone Representation under Low-resource Transfer
by: Kim, Minu, et al.
Published: (2025)

HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition
by: Yoon, Ji Won, et al.
Published: (2022)

Persistent Homology of Music Network with Three Different Distances
by: Heo, Eunwoo, et al.
Published: (2025)

PerformSinger: Multimodal Singing Voice Synthesis Leveraging Synchronized Lip Cues from Singing Performance Videos
by: Gu, Ke, et al.
Published: (2025)

Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Disease
by: Hernandez, Abner, et al.
Published: (2026)

Read, Watch and Scream! Sound Generation from Text and Video
by: Jeong, Yujin, et al.
Published: (2024)