Cover Image

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yadav, Hemant, Sitaram, Sunayana, Shah, Rajiv Ratn
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

JOOCI: a Framework for Learning Comprehensive Speech Representations
by: Yadav, Hemant, et al.
Published: (2024)

Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation
by: Yadav, Hemant, et al.
Published: (2024)

Analysing the Masked predictive coding training criterion for pre-training a Speech Representation Model
by: Yadav, Hemant, et al.
Published: (2023)

HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition
by: Yoon, Ji Won, et al.
Published: (2022)

SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT
by: Cho, Cheol Jun, et al.
Published: (2023)

mHuBERT-147: A Compact Multilingual HuBERT Model
by: Boito, Marcely Zanon, et al.
Published: (2024)

MelHuBERT: A simplified HuBERT on Mel spectrograms
by: Lin, Tzu-Quan, et al.
Published: (2022)

Scaling HuBERT for African Languages: From Base to Large and XL
by: Caubrière, Antoine, et al.
Published: (2025)

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining
by: Vielzeuf, Valentin
Published: (2024)

Target Speech Extraction with Pre-trained AV-HuBERT and Mask-And-Recover Strategy
by: Wu, Wenxuan, et al.
Published: (2024)

Phonetic and Lexical Discovery of a Canine Language using HuBERT
by: Li, Xingyuan, et al.
Published: (2024)

ExHuBERT: Enhancing HuBERT Through Block Extension and Fine-Tuning on 37 Emotion Datasets
by: Amiriparian, Shahin, et al.
Published: (2024)

Self-Supervised Syllable Discovery Based on Speaker-Disentangled HuBERT
by: Komatsu, Ryota, et al.
Published: (2024)

M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks
by: Schneider, Florian, et al.
Published: (2024)

Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0
by: Huo, Robin, et al.
Published: (2025)

Exploring Pretraining via Active Forgetting for Improving Cross Lingual Transfer for Decoder Language Models
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Long-context Non-factoid Question Answering in Indic Languages
by: Mishra, Ritwik, et al.
Published: (2025)

LittiChoQA: Literary Texts in Indic Languages Chosen for Question Answering
by: Khandelwal, Aarya, et al.
Published: (2026)

VisG AV-HuBERT: Viseme-Guided AV-HuBERT
by: Papadopoulos, Aristeidis, et al.
Published: (2026)

A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
by: Sathe, Ashutosh, et al.
Published: (2024)

MaiBERT: A Pre-training Corpus and Language Model for Low-Resourced Maithili Language
by: Yadav, Sumit, et al.
Published: (2025)

MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Artificial Rigidities vs. Biological Noise: A Comparative Analysis of Multisensory Integration in AV-HuBERT and Human Observers
by: López, Francisco Portillo
Published: (2026)

Text-guided HuBERT: Self-Supervised Speech Pre-training via Generative Adversarial Networks
by: Ma, Duo, et al.
Published: (2024)

Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Contamination Report for Multilingual Benchmarks
by: Ahuja, Sanchit, et al.
Published: (2024)

Multilingual Coreference Resolution in Low-resource South Asian Languages
by: Mishra, Ritwik, et al.
Published: (2024)

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates
by: Huang, Zikang, et al.
Published: (2026)

HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings
by: Gumma, Varun, et al.
Published: (2024)

Improving Self Consistency in LLMs through Probabilistic Tokenization
by: Sathe, Ashutosh, et al.
Published: (2024)

Med-CoDE: Medical Critique based Disagreement Evaluation Framework
by: Gupta, Mohit, et al.
Published: (2025)

ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models
by: Chaffin, Antoine, et al.
Published: (2026)

FaBERT: Pre-training BERT on Persian Blogs
by: Masumi, Mostafa, et al.
Published: (2024)

How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion
by: Seth, Agrima, et al.
Published: (2025)

Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction
by: Shi, Jiatong, et al.
Published: (2023)

CultureLLM: Incorporating Cultural Differences into Large Language Models
by: Li, Cheng, et al.
Published: (2024)

DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures
by: Seth, Agrima, et al.
Published: (2024)

Context-Enhanced Language Models for Generating Multi-Paper Citations
by: Anand, Avinash, et al.
Published: (2024)

Structured Definitions and Segmentations for Legal Reasoning in LLMs: A Study on Indian Legal Data
by: Khatri, Mann, et al.
Published: (2025)

MAFIA: Multi-Adapter Fused Inclusive LanguAge Models
by: Jain, Prachi, et al.
Published: (2024)