Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Zijian, Lin, Xiaoxin, Pu, Tao, Yuan, Zhenlong, Wang, Guangrun, Lin, Liang
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866918251571707904
author	Song, Zijian Lin, Xiaoxin Pu, Tao Yuan, Zhenlong Wang, Guangrun Lin, Liang
author_facet	Song, Zijian Lin, Xiaoxin Pu, Tao Yuan, Zhenlong Wang, Guangrun Lin, Liang
contents	Recent progress in robotics and embodied AI is largely driven by Large Multimodal Models (LMMs). However, a key challenge remains underexplored: how can we advance LMMs to discover tasks that assist humans in open-future scenarios, where human intentions are highly concurrent and dynamic. In this work, we formalize the problem of Human-centric Open-future Task Discovery (HOTD), focusing particularly on identifying tasks that reduce human effort across plausible futures. To facilitate this study, we propose HOTD-Bench, which features over 2K real-world videos, a semi-automated annotation pipeline, and a simulation-based protocol tailored for open-set future evaluation. Additionally, we propose the Collaborative Multi-Agent Search Tree (CMAST) framework, which decomposes complex reasoning through a multi-agent system and structures the reasoning process through a scalable search tree module. In our experiments, CMAST achieves the best performance on the HOTD-Bench, significantly surpassing existing LMMs. It also integrates well with existing LMMs, consistently improving performance.
format	Preprint
id	arxiv_https___arxiv_org_abs_2511_18929
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search Song, Zijian Lin, Xiaoxin Pu, Tao Yuan, Zhenlong Wang, Guangrun Lin, Liang Computer Vision and Pattern Recognition Recent progress in robotics and embodied AI is largely driven by Large Multimodal Models (LMMs). However, a key challenge remains underexplored: how can we advance LMMs to discover tasks that assist humans in open-future scenarios, where human intentions are highly concurrent and dynamic. In this work, we formalize the problem of Human-centric Open-future Task Discovery (HOTD), focusing particularly on identifying tasks that reduce human effort across plausible futures. To facilitate this study, we propose HOTD-Bench, which features over 2K real-world videos, a semi-automated annotation pipeline, and a simulation-based protocol tailored for open-set future evaluation. Additionally, we propose the Collaborative Multi-Agent Search Tree (CMAST) framework, which decomposes complex reasoning through a multi-agent system and structures the reasoning process through a scalable search tree module. In our experiments, CMAST achieves the best performance on the HOTD-Bench, significantly surpassing existing LMMs. It also integrates well with existing LMMs, consistently improving performance.
title	Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search
topic	Computer Vision and Pattern Recognition
url	https://arxiv.org/abs/2511.18929