Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.10325 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915023909027840 |
|---|---|
| author | Liu, Joseph Nandwana, Mahesh Kumar Pylkkönen, Janne Heikinheimo, Hannes McGuire, Morgan |
| author_facet | Liu, Joseph Nandwana, Mahesh Kumar Pylkkönen, Janne Heikinheimo, Hannes McGuire, Morgan |
| contents | Toxicity classification for voice heavily relies on the semantic content of speech. We propose a novel framework that utilizes cross-modal learning to integrate the semantic embedding of text into a multilabel speech toxicity classifier during training. This enables us to incorporate textual information during training while still requiring only audio during inference. We evaluate this classifier on large-scale datasets with real-world characteristics to validate the effectiveness of this framework. Through ablation studies, we demonstrate that general-purpose semantic text embeddings are rich and aligned with speech for toxicity classification purposes. Conducting experiments across multiple languages at scale, we show improvements in voice toxicity classification across five languages and different toxicity categories. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10325 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment Liu, Joseph Nandwana, Mahesh Kumar Pylkkönen, Janne Heikinheimo, Hannes McGuire, Morgan Computation and Language Machine Learning Audio and Speech Processing Toxicity classification for voice heavily relies on the semantic content of speech. We propose a novel framework that utilizes cross-modal learning to integrate the semantic embedding of text into a multilabel speech toxicity classifier during training. This enables us to incorporate textual information during training while still requiring only audio during inference. We evaluate this classifier on large-scale datasets with real-world characteristics to validate the effectiveness of this framework. Through ablation studies, we demonstrate that general-purpose semantic text embeddings are rich and aligned with speech for toxicity classification purposes. Conducting experiments across multiple languages at scale, we show improvements in voice toxicity classification across five languages and different toxicity categories. |
| title | Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment |
| topic | Computation and Language Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.10325 |