Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.20116 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909366437806080 |
|---|---|
| author | Lin, Spencer Rizk, Basem Jun, Miru Artze, Andy Sullivan, Caitlin Mozgai, Sharon Fisher, Scott |
| author_facet | Lin, Spencer Rizk, Basem Jun, Miru Artze, Andy Sullivan, Caitlin Mozgai, Sharon Fisher, Scott |
| contents | The rise in capability and ubiquity of generative artificial intelligence (AI) technologies has enabled its application to the field of Socially Interactive Agents (SIAs). Despite rising interest in modern AI-powered components used for real-time SIA research, substantial friction remains due to the absence of a standardized and universal SIA framework. To target this absence, we developed Estuary: a multimodal (text, audio, and soon video) framework which facilitates the development of low-latency, real-time SIAs. Estuary seeks to reduce repeat work between studies and to provide a flexible platform that can be run entirely off-cloud to maximize configurability, controllability, reproducibility of studies, and speed of agent response times. We are able to do this by constructing a robust multimodal framework which incorporates current and future components seamlessly into a modular and interoperable architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20116 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Estuary: A Framework For Building Multimodal Low-Latency Real-Time Socially Interactive Agents Lin, Spencer Rizk, Basem Jun, Miru Artze, Andy Sullivan, Caitlin Mozgai, Sharon Fisher, Scott Human-Computer Interaction Artificial Intelligence J.0 The rise in capability and ubiquity of generative artificial intelligence (AI) technologies has enabled its application to the field of Socially Interactive Agents (SIAs). Despite rising interest in modern AI-powered components used for real-time SIA research, substantial friction remains due to the absence of a standardized and universal SIA framework. To target this absence, we developed Estuary: a multimodal (text, audio, and soon video) framework which facilitates the development of low-latency, real-time SIAs. Estuary seeks to reduce repeat work between studies and to provide a flexible platform that can be run entirely off-cloud to maximize configurability, controllability, reproducibility of studies, and speed of agent response times. We are able to do this by constructing a robust multimodal framework which incorporates current and future components seamlessly into a modular and interoperable architecture. |
| title | Estuary: A Framework For Building Multimodal Low-Latency Real-Time Socially Interactive Agents |
| topic | Human-Computer Interaction Artificial Intelligence J.0 |
| url | https://arxiv.org/abs/2410.20116 |