Exploring How Audio Effects Alter Emotion with Foundation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913151486787584 |
|---|---|
| author | Katsis, Stelios Lyberatos, Vassilis Kantarelis, Spyridon Dervakos, Edmund Stamou, Giorgos |
| author_facet | Katsis, Stelios Lyberatos, Vassilis Kantarelis, Spyridon Dervakos, Edmund Stamou, Giorgos |
| contents | Audio effects (FX) such as reverberation, distortion, modulation, and dynamic range processing play a pivotal role in shaping emotional responses during music listening. While prior studies have examined links between low-level audio features and affective perception, the systematic impact of audio FX on emotion remains underexplored. This work investigates how foundation models - large-scale neural architectures pretrained on multimodal data - can be leveraged to analyze these effects. Such models encode rich associations between musical structure, timbre, and affective meaning, offering a powerful framework for probing the emotional consequences of sound design techniques. By applying various probing methods to embeddings from deep learning models, we examine the complex, nonlinear relationships between audio FX and estimated emotion, uncovering patterns tied to specific effects and evaluating the robustness of foundation audio models. Our findings aim to advance understanding of the perceptual impact of audio production practices, with implications for music cognition, performance, and affective computing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15151 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring How Audio Effects Alter Emotion with Foundation Models Katsis, Stelios Lyberatos, Vassilis Kantarelis, Spyridon Dervakos, Edmund Stamou, Giorgos Sound Artificial Intelligence Audio effects (FX) such as reverberation, distortion, modulation, and dynamic range processing play a pivotal role in shaping emotional responses during music listening. While prior studies have examined links between low-level audio features and affective perception, the systematic impact of audio FX on emotion remains underexplored. This work investigates how foundation models - large-scale neural architectures pretrained on multimodal data - can be leveraged to analyze these effects. Such models encode rich associations between musical structure, timbre, and affective meaning, offering a powerful framework for probing the emotional consequences of sound design techniques. By applying various probing methods to embeddings from deep learning models, we examine the complex, nonlinear relationships between audio FX and estimated emotion, uncovering patterns tied to specific effects and evaluating the robustness of foundation audio models. Our findings aim to advance understanding of the perceptual impact of audio production practices, with implications for music cognition, performance, and affective computing. |
| title | Exploring How Audio Effects Alter Emotion with Foundation Models |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2509.15151 |