LLAMAPIE: Proactive In-Ear Conversation Assistants
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916867732406272 |
|---|---|
| author | Chen, Tuochao Batchelder, Nicholas Liu, Alisa Smith, Noah Gollakota, Shyamnath |
| author_facet | Chen, Tuochao Batchelder, Nicholas Liu, Alisa Smith, Noah Gollakota, Shyamnath |
| contents | We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike traditional language models that require explicit user invocation, this assistant operates in the background, anticipating user needs without interrupting conversations. We address several challenges, including determining when to respond, crafting concise responses that enhance conversations, leveraging knowledge of the user for context-aware assistance, and real-time, on-device processing. To achieve this, we construct a semi-synthetic dialogue dataset and propose a two-model pipeline: a small model that decides when to respond and a larger model that generates the response. We evaluate our approach on real-world datasets, demonstrating its effectiveness in providing helpful, unobtrusive assistance. User studies with our assistant, implemented on Apple Silicon M2 hardware, show a strong preference for the proactive assistant over both a baseline with no assistance and a reactive model, highlighting the potential of LlamaPie to enhance live conversations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_04066 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLAMAPIE: Proactive In-Ear Conversation Assistants Chen, Tuochao Batchelder, Nicholas Liu, Alisa Smith, Noah Gollakota, Shyamnath Machine Learning Computation and Language Human-Computer Interaction Sound Audio and Speech Processing We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike traditional language models that require explicit user invocation, this assistant operates in the background, anticipating user needs without interrupting conversations. We address several challenges, including determining when to respond, crafting concise responses that enhance conversations, leveraging knowledge of the user for context-aware assistance, and real-time, on-device processing. To achieve this, we construct a semi-synthetic dialogue dataset and propose a two-model pipeline: a small model that decides when to respond and a larger model that generates the response. We evaluate our approach on real-world datasets, demonstrating its effectiveness in providing helpful, unobtrusive assistance. User studies with our assistant, implemented on Apple Silicon M2 hardware, show a strong preference for the proactive assistant over both a baseline with no assistance and a reactive model, highlighting the potential of LlamaPie to enhance live conversations. |
| title | LLAMAPIE: Proactive In-Ear Conversation Assistants |
| topic | Machine Learning Computation and Language Human-Computer Interaction Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.04066 |