Segmental Attention Decoding With Long Form Acoustic Encodings
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914204609413120 |
|---|---|
| author | Swietojanski, Pawel Li, Xinwei Xu, Mingbin Hori, Takaaki Can, Dogan Zhuang, Xiaodan |
| author_facet | Swietojanski, Pawel Li, Xinwei Xu, Mingbin Hori, Takaaki Can, Dogan Zhuang, Xiaodan |
| contents | We address the fundamental incompatibility of attention-based encoder-decoder (AED) models with long-form acoustic encodings. AED models trained on segmented utterances learn to encode absolute frame positions by exploiting limited acoustic context beyond segment boundaries, but fail to generalize when decoding long-form segments where these cues vanish. The model loses ability to order acoustic encodings due to permutation invariance of keys and values in cross-attention. We propose four modifications: (1) injecting explicit absolute positional encodings into cross-attention for each decoded segment, (2) long-form training with extended acoustic context to eliminate implicit absolute position encoding, (3) segment concatenation to cover diverse segmentations needed during training, and (4) semantic segmentation to align AED-decoded segments with training segments. We show these modifications close the accuracy gap between continuous and segmented acoustic encodings, enabling auto-regressive use of the attention decoder. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14652 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Segmental Attention Decoding With Long Form Acoustic Encodings Swietojanski, Pawel Li, Xinwei Xu, Mingbin Hori, Takaaki Can, Dogan Zhuang, Xiaodan Audio and Speech Processing Computation and Language We address the fundamental incompatibility of attention-based encoder-decoder (AED) models with long-form acoustic encodings. AED models trained on segmented utterances learn to encode absolute frame positions by exploiting limited acoustic context beyond segment boundaries, but fail to generalize when decoding long-form segments where these cues vanish. The model loses ability to order acoustic encodings due to permutation invariance of keys and values in cross-attention. We propose four modifications: (1) injecting explicit absolute positional encodings into cross-attention for each decoded segment, (2) long-form training with extended acoustic context to eliminate implicit absolute position encoding, (3) segment concatenation to cover diverse segmentations needed during training, and (4) semantic segmentation to align AED-decoded segments with training segments. We show these modifications close the accuracy gap between continuous and segmented acoustic encodings, enabling auto-regressive use of the attention decoder. |
| title | Segmental Attention Decoding With Long Form Acoustic Encodings |
| topic | Audio and Speech Processing Computation and Language |
| url | https://arxiv.org/abs/2512.14652 |