A Musicolinguistic Approach to Data Sonification in Real-Time Immersive Systems
Fuente:
Zenodo
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901506144337920 |
|---|---|
| author | Gülay, Saim |
| author_facet | Gülay, Saim |
| contents | <p>This deposit contains the MSc dissertation <em>“A Musicolinguistic Approach to Data Sonification in Real-Time Immersive Systems”</em> (University of Nottingham, MSc Human–Computer Interaction, 2025).</p> <p>The work proposes a musico-linguistic framework for sonification in which data–sound mappings are treated as a “Conceptual Structure” combining phonetic, syntactic, and semantic layers, following musicolinguistic and linguistic theory. Rather than focusing only on low-level timbre or abstract pitch mappings, the study systematically tests how phonetic (formant-based timbre), syntactic (tetrachord-based melodic structures), and semantic (tonal cadences) layers – alone and in combination – shape user experience.</p> <p>To evaluate this framework, a real-time Unity-based simulation of <em>Escherichia coli</em> in an eight-zone environment was developed. Four genomic parameters (optimal temperature, optimal pH, UV resistance, toxin resistance) are continuously sonified under five conditions: phonetic, semantic, syntactic, a Hybrid “Conceptual Structure” mapping (all three layers combined), and a silent control. The audio synthesis chain is held constant; only the mapping logic changes between conditions. A within-subjects experiment (n = 10) measures immersion/engagement (IEQ-SF), mental workload (NASA-TLX), an EEG-derived attention index, and user preferences. All states are recorded in a unified JSON log, with person-specific EEG baselines and artefact veto policies to support reproducible analysis.</p> <p>Results indicate that the Hybrid/Conceptual Structure condition outperforms phonetic and syntactic mappings on immersion and is clearly preferred by participants, while workload and EEG-based attention show no significant differences under the current small sample and short EEG windows. The study is positioned as a pilot: its main contributions are (i) a musico-linguistic conceptual structure framework for sonification, (ii) a comparative real-time testbed for phonetic/syntactic/semantic mappings under a fixed synthesis chain, and (iii) a reproducible design template for mixed-reality sonification experiments, intended to be extended in future work with larger samples and richer musicolinguistic toolsets.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17750142 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | A Musicolinguistic Approach to Data Sonification in Real-Time Immersive Systems Gülay, Saim Sonication/statistics & numerical data Data Sonification Musicolinguistic <p>This deposit contains the MSc dissertation <em>“A Musicolinguistic Approach to Data Sonification in Real-Time Immersive Systems”</em> (University of Nottingham, MSc Human–Computer Interaction, 2025).</p> <p>The work proposes a musico-linguistic framework for sonification in which data–sound mappings are treated as a “Conceptual Structure” combining phonetic, syntactic, and semantic layers, following musicolinguistic and linguistic theory. Rather than focusing only on low-level timbre or abstract pitch mappings, the study systematically tests how phonetic (formant-based timbre), syntactic (tetrachord-based melodic structures), and semantic (tonal cadences) layers – alone and in combination – shape user experience.</p> <p>To evaluate this framework, a real-time Unity-based simulation of <em>Escherichia coli</em> in an eight-zone environment was developed. Four genomic parameters (optimal temperature, optimal pH, UV resistance, toxin resistance) are continuously sonified under five conditions: phonetic, semantic, syntactic, a Hybrid “Conceptual Structure” mapping (all three layers combined), and a silent control. The audio synthesis chain is held constant; only the mapping logic changes between conditions. A within-subjects experiment (n = 10) measures immersion/engagement (IEQ-SF), mental workload (NASA-TLX), an EEG-derived attention index, and user preferences. All states are recorded in a unified JSON log, with person-specific EEG baselines and artefact veto policies to support reproducible analysis.</p> <p>Results indicate that the Hybrid/Conceptual Structure condition outperforms phonetic and syntactic mappings on immersion and is clearly preferred by participants, while workload and EEG-based attention show no significant differences under the current small sample and short EEG windows. The study is positioned as a pilot: its main contributions are (i) a musico-linguistic conceptual structure framework for sonification, (ii) a comparative real-time testbed for phonetic/syntactic/semantic mappings under a fixed synthesis chain, and (iii) a reproducible design template for mixed-reality sonification experiments, intended to be extended in future work with larger samples and richer musicolinguistic toolsets.</p> |
| title | A Musicolinguistic Approach to Data Sonification in Real-Time Immersive Systems |
| topic | Sonication/statistics & numerical data Data Sonification Musicolinguistic |
| url | https://doi.org/10.5281/zenodo.17750142 |