DroidRetriever: A Transparent and Steerable Automation System for Collaborative Mobile Information Seeking
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910118862389248 |
|---|---|
| author | Bian, Yiheng Song, Yunpeng Ma, Guiyu Zhu, Rongrong Cai, Zhongmin |
| author_facet | Bian, Yiheng Song, Yunpeng Ma, Guiyu Zhu, Rongrong Cai, Zhongmin |
| contents | Information seeking on mobile devices is often fragmented, trapping users in repetitive cycles of context switching and data re-entry, which increases cognitive load and disrupts workflow. Existing mobile agents provide limited cross-source integration and are largely opaque, presenting progress as a linear feed with few opportunities to intervene, steer, or take control. We present DroidRetriever, a transparent, steerable system for cross-source mobile information seeking. It accepts voice or typed input and the multi-LLM system decomposes the task, navigates to target pages, takes screenshots, and synthesizes a concise report with citation-linked screenshots. We make the process transparent through a progress dashboard combining sub-task progress and real-time exploration maps for seamless takeover. DroidRetriever also pauses on detected privacy or high-risk screens and prompts intervention. Across 35 tasks over 24 apps, experiments and user studies demonstrate improvements in coverage, transparency, and reduced workload. We release our code at https://github.com/AkimotoAyako/DroidRetriever. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_03364 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DroidRetriever: A Transparent and Steerable Automation System for Collaborative Mobile Information Seeking Bian, Yiheng Song, Yunpeng Ma, Guiyu Zhu, Rongrong Cai, Zhongmin Human-Computer Interaction H.5.2; H.3.3 Information seeking on mobile devices is often fragmented, trapping users in repetitive cycles of context switching and data re-entry, which increases cognitive load and disrupts workflow. Existing mobile agents provide limited cross-source integration and are largely opaque, presenting progress as a linear feed with few opportunities to intervene, steer, or take control. We present DroidRetriever, a transparent, steerable system for cross-source mobile information seeking. It accepts voice or typed input and the multi-LLM system decomposes the task, navigates to target pages, takes screenshots, and synthesizes a concise report with citation-linked screenshots. We make the process transparent through a progress dashboard combining sub-task progress and real-time exploration maps for seamless takeover. DroidRetriever also pauses on detected privacy or high-risk screens and prompts intervention. Across 35 tasks over 24 apps, experiments and user studies demonstrate improvements in coverage, transparency, and reduced workload. We release our code at https://github.com/AkimotoAyako/DroidRetriever. |
| title | DroidRetriever: A Transparent and Steerable Automation System for Collaborative Mobile Information Seeking |
| topic | Human-Computer Interaction H.5.2; H.3.3 |
| url | https://arxiv.org/abs/2505.03364 |