Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.27957 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914521258393600 |
|---|---|
| author | Mermerci, Mert Pascoe, Emile Edström, Fredrik Kjellström, Hedvig |
| author_facet | Mermerci, Mert Pascoe, Emile Edström, Fredrik Kjellström, Hedvig |
| contents | We present a museum installation in a 180° dome theater, which gives the museum visitor the experience of conducting a symphony orchestra. We have pre-recorded a short music piece performed by a professional orchestra. This recording is played back in the dome with the visitor standing in the conductor's position. The visitor's gestures are captured with a vision-based skeleton tracker, steering the recording playback pace via a gesture recognition module that translates the gestures into a time control signal. This is sent to a playback module that plays the recording in the dome at the corresponding speed. The gesture recognition module is based on a hierarchical LSTM network, trained with recorded sequences of multiple conductors with different level of expertise conducting the same recording. The system is evaluated with a quantitative study of the estimated timing accuracy, a user study evaluating the musical realism and usability of the real-time control, and a field study to evaluate the performance of the entire system with real museum visitors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_27957 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Real-Time Control of a Virtual Orchestra by Recognition of Conducting Gestures Mermerci, Mert Pascoe, Emile Edström, Fredrik Kjellström, Hedvig Human-Computer Interaction We present a museum installation in a 180° dome theater, which gives the museum visitor the experience of conducting a symphony orchestra. We have pre-recorded a short music piece performed by a professional orchestra. This recording is played back in the dome with the visitor standing in the conductor's position. The visitor's gestures are captured with a vision-based skeleton tracker, steering the recording playback pace via a gesture recognition module that translates the gestures into a time control signal. This is sent to a playback module that plays the recording in the dome at the corresponding speed. The gesture recognition module is based on a hierarchical LSTM network, trained with recorded sequences of multiple conductors with different level of expertise conducting the same recording. The system is evaluated with a quantitative study of the estimated timing accuracy, a user study evaluating the musical realism and usability of the real-time control, and a field study to evaluate the performance of the entire system with real museum visitors. |
| title | Real-Time Control of a Virtual Orchestra by Recognition of Conducting Gestures |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2604.27957 |