PAM: Prompting Audio-Language Models for Audio Quality Assessment
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929230784233472 |
|---|---|
| author | Deshmukh, Soham Alharthi, Dareen Elizalde, Benjamin Gamper, Hannes Ismail, Mahmoud Al Singh, Rita Raj, Bhiksha Wang, Huaming |
| author_facet | Deshmukh, Soham Alharthi, Dareen Elizalde, Benjamin Gamper, Hannes Ismail, Mahmoud Al Singh, Rita Raj, Bhiksha Wang, Huaming |
| contents | While audio quality is a key performance metric for various audio processing tasks, including generative modeling, its objective measurement remains a challenge. Audio-Language Models (ALMs) are pre-trained on audio-text pairs that may contain information about audio quality, the presence of artifacts, or noise. Given an audio input and a text prompt related to quality, an ALM can be used to calculate a similarity score between the two. Here, we exploit this capability and introduce PAM, a no-reference metric for assessing audio quality for different audio processing tasks. Contrary to other "reference-free" metrics, PAM does not require computing embeddings on a reference dataset nor training a task-specific model on a costly set of human listening scores. We extensively evaluate the reliability of PAM against established metrics and human listening scores on four tasks: text-to-audio (TTA), text-to-music generation (TTM), text-to-speech (TTS), and deep noise suppression (DNS). We perform multiple ablation studies with controlled distortions, in-the-wild setups, and prompt choices. Our evaluation shows that PAM correlates well with existing metrics and human listening scores. These results demonstrate the potential of ALMs for computing a general-purpose audio quality metric. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00282 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PAM: Prompting Audio-Language Models for Audio Quality Assessment Deshmukh, Soham Alharthi, Dareen Elizalde, Benjamin Gamper, Hannes Ismail, Mahmoud Al Singh, Rita Raj, Bhiksha Wang, Huaming Audio and Speech Processing Sound While audio quality is a key performance metric for various audio processing tasks, including generative modeling, its objective measurement remains a challenge. Audio-Language Models (ALMs) are pre-trained on audio-text pairs that may contain information about audio quality, the presence of artifacts, or noise. Given an audio input and a text prompt related to quality, an ALM can be used to calculate a similarity score between the two. Here, we exploit this capability and introduce PAM, a no-reference metric for assessing audio quality for different audio processing tasks. Contrary to other "reference-free" metrics, PAM does not require computing embeddings on a reference dataset nor training a task-specific model on a costly set of human listening scores. We extensively evaluate the reliability of PAM against established metrics and human listening scores on four tasks: text-to-audio (TTA), text-to-music generation (TTM), text-to-speech (TTS), and deep noise suppression (DNS). We perform multiple ablation studies with controlled distortions, in-the-wild setups, and prompt choices. Our evaluation shows that PAM correlates well with existing metrics and human listening scores. These results demonstrate the potential of ALMs for computing a general-purpose audio quality metric. |
| title | PAM: Prompting Audio-Language Models for Audio Quality Assessment |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2402.00282 |