OmniGAIA: Towards Native Omni-Modal AI Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914361156567040 |
|---|---|
| author | Li, Xiaoxi Jiao, Wenxiang Jin, Jiarui Wang, Shijian Dong, Guanting Jin, Jiajie Wang, Hao Wang, Yinuo Wen, Ji-Rong Lu, Yuan Dou, Zhicheng |
| author_facet | Li, Xiaoxi Jiao, Wenxiang Jin, Jiarui Wang, Shijian Dong, Guanting Jin, Jiajie Wang, Hao Wang, Yinuo Wen, Ji-Rong Lu, Yuan Dou, Zhicheng |
| contents | Human intelligence naturally intertwines omni-modal perception -- spanning vision, audio, and language -- with complex reasoning and tool usage to interact with the world. However, current multi-modal LLMs are primarily confined to bi-modal interactions (e.g., vision-language), lacking the unified cognitive capabilities required for general AI assistants. To bridge this gap, we introduce OmniGAIA, a comprehensive benchmark designed to evaluate omni-modal agents on tasks necessitating deep reasoning and multi-turn tool execution across video, audio, and image modalities. Constructed via a novel omni-modal event graph approach, OmniGAIA synthesizes complex, multi-hop queries derived from real-world data that require cross-modal reasoning and external tool integration. Furthermore, we propose OmniAtlas, a native omni-modal foundation agent under tool-integrated reasoning paradigm with active omni-modal perception. Trained on trajectories synthesized via a hindsight-guided tree exploration strategy and OmniDPO for fine-grained error correction, OmniAtlas effectively enhances the tool-use capabilities of existing open-source models. This work marks a step towards next-generation native omni-modal AI assistants for real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_22897 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniGAIA: Towards Native Omni-Modal AI Agents Li, Xiaoxi Jiao, Wenxiang Jin, Jiarui Wang, Shijian Dong, Guanting Jin, Jiajie Wang, Hao Wang, Yinuo Wen, Ji-Rong Lu, Yuan Dou, Zhicheng Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning Multimedia Human intelligence naturally intertwines omni-modal perception -- spanning vision, audio, and language -- with complex reasoning and tool usage to interact with the world. However, current multi-modal LLMs are primarily confined to bi-modal interactions (e.g., vision-language), lacking the unified cognitive capabilities required for general AI assistants. To bridge this gap, we introduce OmniGAIA, a comprehensive benchmark designed to evaluate omni-modal agents on tasks necessitating deep reasoning and multi-turn tool execution across video, audio, and image modalities. Constructed via a novel omni-modal event graph approach, OmniGAIA synthesizes complex, multi-hop queries derived from real-world data that require cross-modal reasoning and external tool integration. Furthermore, we propose OmniAtlas, a native omni-modal foundation agent under tool-integrated reasoning paradigm with active omni-modal perception. Trained on trajectories synthesized via a hindsight-guided tree exploration strategy and OmniDPO for fine-grained error correction, OmniAtlas effectively enhances the tool-use capabilities of existing open-source models. This work marks a step towards next-generation native omni-modal AI assistants for real-world scenarios. |
| title | OmniGAIA: Towards Native Omni-Modal AI Agents |
| topic | Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2602.22897 |