Cover Image

Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gaido, Marco, Papi, Sara, Negri, Matteo, Bentivogli, Luisa
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena
by: Gaido, Marco, et al.
Published: (2024)

SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation
by: Papi, Sara, et al.
Published: (2024)

StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection
by: Papi, Sara, et al.
Published: (2024)

Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems
by: Gaido, Marco, et al.
Published: (2025)

How to Evaluate Speech Translation with Source-Aware Neural MT Metrics
by: Cettolo, Mauro, et al.
Published: (2025)

Cross-Attention is Half Explanation in Speech-to-Text Models
by: Papi, Sara, et al.
Published: (2025)

SimulU: Training-free Policy for Long-form Simultaneous Speech-to-Speech Translation
by: Djanibekov, Amirbek, et al.
Published: (2026)

Different Speech Translation Models Encode and Translate Speaker Gender Differently
by: Fucci, Dennis, et al.
Published: (2025)

FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian
by: Papi, Sara, et al.
Published: (2025)

MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages
by: Gaido, Marco, et al.
Published: (2024)

How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not
by: Verdini, Francesco, et al.
Published: (2024)

SBAAM! Eliminating Transcript Dependency in Automatic Subtitling
by: Gaido, Marco, et al.
Published: (2024)

The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence
by: Gaido, Marco, et al.
Published: (2025)

What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study
by: Savoldi, Beatrice, et al.
Published: (2024)

Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
by: Conti, Lina, et al.
Published: (2025)

DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
by: Papi, Sara, et al.
Published: (2026)

The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models
by: Conti, Lina, et al.
Published: (2025)

Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison
by: Lam, Tsz Kin, et al.
Published: (2025)

SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation
by: Fucci, Dennis, et al.
Published: (2024)

AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation
by: Papi, Sara, et al.
Published: (2023)

Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Models
by: Piergentili, Andrea, et al.
Published: (2024)

Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation
by: Polák, Peter, et al.
Published: (2025)

Speech Foundation Models and Crowdsourcing for Efficient, High-Quality Data Collection
by: Lee, Beomseok, et al.
Published: (2024)

When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP
by: Papi, Sara, et al.
Published: (2023)

Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attribution
by: Fucci, Dennis, et al.
Published: (2025)

NUTSHELL: A Dataset for Abstract Generation from Scientific Talks
by: Züfle, Maike, et al.
Published: (2025)

Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond
by: Lee, Beomseok, et al.
Published: (2024)

Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions
by: Savoldi, Beatrice, et al.
Published: (2025)

An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation
by: Piergentili, Andrea, et al.
Published: (2025)

Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
by: Piergentili, Andrea, et al.
Published: (2025)

A Prompt Response to the Demand for Automatic Gender-Neutral Translation
by: Savoldi, Beatrice, et al.
Published: (2024)

Do What I Say: A Spoken Prompt Dataset for Instruction-Following
by: Züfle, Maike, et al.
Published: (2026)

MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks
by: Papi, Sara, et al.
Published: (2025)

Granary: Speech Recognition and Translation Dataset in 25 European Languages
by: Koluguri, Nithin Rao, et al.
Published: (2025)

What Do Speech Foundation Models Not Learn About Speech?
by: Waheed, Abdul, et al.
Published: (2024)

What do Speech Foundation Models Learn? Analysis and Applications
by: Pasad, Ankita
Published: (2025)

"Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most
by: Zhou, Kaitlyn, et al.
Published: (2026)

End-to-end Automatic Speech Recognition and Speech Translation: Integration of Speech Foundational Models and LLMs
by: Luu, Nam, et al.
Published: (2025)

Speech Translation Refinement using Large Language Models
by: Dou, Huaixia, et al.
Published: (2025)

How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?
by: Papi, Sara, et al.
Published: (2024)