Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.01934 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912409693716480 |
|---|---|
| author | Yao, Yiqun Li, Xiang Jiang, Xin Fang, Xuezhi Yu, Naitong Sun, Aixin Wang, Yequan |
| author_facet | Yao, Yiqun Li, Xiang Jiang, Xin Fang, Xuezhi Yu, Naitong Sun, Aixin Wang, Yequan |
| contents | Humans naturally process real-world multimodal information in a full-duplex manner. In artificial intelligence, replicating this capability is essential for advancing model development and deployment, particularly in embodied contexts. The development of multimodal models faces two primary challenges: (1) effectively handling more than three modalities-such as vision, audio, and text; and (2) delivering full-duplex responses to rapidly evolving human instructions. To facilitate research on models that support both omnimodal processing and full duplexity, we present RoboEgo (alias: FLM-Ego), a unified model system designed to address both challenges. RoboEgo incorporates a backbone architecture and algorithms that natively support full duplexity, achieving a theoretical duplex latency of 80 ms. In streaming visually grounded conversations under real-world conditions, RoboEgo exhibits superior responsiveness and speech naturalness, while maintaining comparable content qualities to state-of-the-art semi-duplex omnimodal models-a feat previously considered unattainable by native full-duplex systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01934 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RoboEgo System Card: An Omnimodal Model with Native Full Duplexity Yao, Yiqun Li, Xiang Jiang, Xin Fang, Xuezhi Yu, Naitong Sun, Aixin Wang, Yequan Artificial Intelligence Humans naturally process real-world multimodal information in a full-duplex manner. In artificial intelligence, replicating this capability is essential for advancing model development and deployment, particularly in embodied contexts. The development of multimodal models faces two primary challenges: (1) effectively handling more than three modalities-such as vision, audio, and text; and (2) delivering full-duplex responses to rapidly evolving human instructions. To facilitate research on models that support both omnimodal processing and full duplexity, we present RoboEgo (alias: FLM-Ego), a unified model system designed to address both challenges. RoboEgo incorporates a backbone architecture and algorithms that natively support full duplexity, achieving a theoretical duplex latency of 80 ms. In streaming visually grounded conversations under real-world conditions, RoboEgo exhibits superior responsiveness and speech naturalness, while maintaining comparable content qualities to state-of-the-art semi-duplex omnimodal models-a feat previously considered unattainable by native full-duplex systems. |
| title | RoboEgo System Card: An Omnimodal Model with Native Full Duplexity |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.01934 |