ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912252600254464 |
|---|---|
| author | Scarlatos, Alexander Wu, Yusong Simon, Ian Roberts, Adam Cooijmans, Tim Jaques, Natasha Tarakajian, Cassie Huang, Cheng-Zhi Anna |
| author_facet | Scarlatos, Alexander Wu, Yusong Simon, Ian Roberts, Adam Cooijmans, Tim Jaques, Natasha Tarakajian, Cassie Huang, Cheng-Zhi Anna |
| contents | Recent advances in generative artificial intelligence (AI) have created models capable of high-quality musical content generation. However, little consideration is given to how to use these models for real-time or cooperative jamming musical applications because of crucial required features: low latency, the ability to communicate planned actions, and the ability to adapt to user input in real-time. To support these needs, we introduce ReaLJam, an interface and protocol for live musical jamming sessions between a human and a Transformer-based AI agent trained with reinforcement learning. We enable real-time interactions using the concept of anticipation, where the agent continually predicts how the performance will unfold and visually conveys its plan to the user. We conduct a user study where experienced musicians jam in real-time with the agent through ReaLJam. Our results demonstrate that ReaLJam enables enjoyable and musically interesting sessions, and we uncover important takeaways for future work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_21267 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers Scarlatos, Alexander Wu, Yusong Simon, Ian Roberts, Adam Cooijmans, Tim Jaques, Natasha Tarakajian, Cassie Huang, Cheng-Zhi Anna Human-Computer Interaction Artificial Intelligence Recent advances in generative artificial intelligence (AI) have created models capable of high-quality musical content generation. However, little consideration is given to how to use these models for real-time or cooperative jamming musical applications because of crucial required features: low latency, the ability to communicate planned actions, and the ability to adapt to user input in real-time. To support these needs, we introduce ReaLJam, an interface and protocol for live musical jamming sessions between a human and a Transformer-based AI agent trained with reinforcement learning. We enable real-time interactions using the concept of anticipation, where the agent continually predicts how the performance will unfold and visually conveys its plan to the user. We conduct a user study where experienced musicians jam in real-time with the agent through ReaLJam. Our results demonstrate that ReaLJam enables enjoyable and musically interesting sessions, and we uncover important takeaways for future work. |
| title | ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformers |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2502.21267 |