herakoi: a sonification experiment for astronomical data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929627480457216 |
|---|---|
| author | Ginolfi, Michele Di Mascolo, Luca Zanella, Anita |
| author_facet | Ginolfi, Michele Di Mascolo, Luca Zanella, Anita |
| contents | Recent research is revealing data-sonification as a promising complementary approach to vision, benefiting both data perception and interpretation. We present herakoi, a novel open-source software that uses machine learning to allow real-time image sonification, with a focus on astronomical data. By tracking hand movements via a webcam and mapping them to image coordinates, herakoi translates visual properties into sound, enabling users to "hear" images. Its swift responsiveness allows users to access information in astronomical images with short training, demonstrating high reliability and effectiveness. The software has shown promise in educational and outreach settings, making complex astronomical concepts more engaging and accessible to diverse audiences, including blind and visually impaired individuals. We also discuss future developments, such as the integration of large language and vision models to create a more interactive experience in interpreting astronomical data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_09152 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | herakoi: a sonification experiment for astronomical data Ginolfi, Michele Di Mascolo, Luca Zanella, Anita Instrumentation and Methods for Astrophysics Human-Computer Interaction Physics Education Recent research is revealing data-sonification as a promising complementary approach to vision, benefiting both data perception and interpretation. We present herakoi, a novel open-source software that uses machine learning to allow real-time image sonification, with a focus on astronomical data. By tracking hand movements via a webcam and mapping them to image coordinates, herakoi translates visual properties into sound, enabling users to "hear" images. Its swift responsiveness allows users to access information in astronomical images with short training, demonstrating high reliability and effectiveness. The software has shown promise in educational and outreach settings, making complex astronomical concepts more engaging and accessible to diverse audiences, including blind and visually impaired individuals. We also discuss future developments, such as the integration of large language and vision models to create a more interactive experience in interpreting astronomical data. |
| title | herakoi: a sonification experiment for astronomical data |
| topic | Instrumentation and Methods for Astrophysics Human-Computer Interaction Physics Education |
| url | https://arxiv.org/abs/2412.09152 |