Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917169698177024 |
|---|---|
| author | Pan, Shuchang Banerjee, Siddharth Hebbar, Dhruv Patel, Siddhant Gupta, Akshaj Cheng, Kan Jen Kim, Hanjo Li, Zeyi Austin Ma, Martin Q. Li, Tingle Anumanchipalli, Gopala Lian, Jiachen |
| author_facet | Pan, Shuchang Banerjee, Siddharth Hebbar, Dhruv Patel, Siddhant Gupta, Akshaj Cheng, Kan Jen Kim, Hanjo Li, Zeyi Austin Ma, Martin Q. Li, Tingle Anumanchipalli, Gopala Lian, Jiachen |
| contents | Human conversation is organized by an implicit chain of thoughts that manifests as timed speech acts. Capturing this causal pathway is key to building natural full-duplex interactive systems. We introduce a framework that enables reasoning over conversational behaviors by modeling this process as causal inference within a Graph-of-Thoughts (GoT). Our approach formalizes the intent-to-action pathway with a hierarchical labeling scheme, predicting high-level communicative intents and low-level speech acts to learn their causal and temporal dependencies. To train this system, we develop a hybrid corpus that pairs controllable, event-rich simulations with human-annotated rationales and real conversational speech. The GoT framework structures streaming predictions as an evolving graph, enabling a multimodal transformer to forecast the next speech act, generate concise justifications for its decisions, and dynamically refine its reasoning. Experiments on both synthetic and real duplex dialogues show that the framework delivers robust behavior detection, produces interpretable reasoning chains, and establishes a foundation for benchmarking conversational reasoning in full duplex spoken dialogue systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21706 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech Pan, Shuchang Banerjee, Siddharth Hebbar, Dhruv Patel, Siddhant Gupta, Akshaj Cheng, Kan Jen Kim, Hanjo Li, Zeyi Austin Ma, Martin Q. Li, Tingle Anumanchipalli, Gopala Lian, Jiachen Computation and Language Artificial Intelligence Human conversation is organized by an implicit chain of thoughts that manifests as timed speech acts. Capturing this causal pathway is key to building natural full-duplex interactive systems. We introduce a framework that enables reasoning over conversational behaviors by modeling this process as causal inference within a Graph-of-Thoughts (GoT). Our approach formalizes the intent-to-action pathway with a hierarchical labeling scheme, predicting high-level communicative intents and low-level speech acts to learn their causal and temporal dependencies. To train this system, we develop a hybrid corpus that pairs controllable, event-rich simulations with human-annotated rationales and real conversational speech. The GoT framework structures streaming predictions as an evolving graph, enabling a multimodal transformer to forecast the next speech act, generate concise justifications for its decisions, and dynamically refine its reasoning. Experiments on both synthetic and real duplex dialogues show that the framework delivers robust behavior detection, produces interpretable reasoning chains, and establishes a foundation for benchmarking conversational reasoning in full duplex spoken dialogue systems. |
| title | Enabling Conversational Behavior Reasoning Capabilities in Full-Duplex Speech |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2512.21706 |