MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912590501773312 |
|---|---|
| author | Bian, Zhipeng Zhu, Jieming Xie, Xuyang Dai, Quanyu Zhao, Zhou Dong, Zhenhua |
| author_facet | Bian, Zhipeng Zhu, Jieming Xie, Xuyang Dai, Quanyu Zhao, Zhou Dong, Zhenhua |
| contents | The rapid advancement of generative AI technologies is driving the integration of diverse AI-powered services into smartphones, transforming how users interact with their devices. To simplify access to predefined AI services, this paper introduces MIRA, a pioneering framework for task instruction recommendation that enables intuitive one-touch AI tasking on smartphones. With MIRA, users can long-press on images or text objects to receive contextually relevant instruction recommendations for executing AI tasks. Our work introduces three key innovations: 1) A multimodal large language model (MLLM)-based recommendation pipeline with structured reasoning to extract key entities, infer user intent, and generate precise instructions; 2) A template-augmented reasoning mechanism that integrates high-level reasoning templates, enhancing task inference accuracy; 3) A prefix-tree-based constrained decoding strategy that restricts outputs to predefined instruction candidates, ensuring coherent and intent-aligned suggestions. Through evaluation using a real-world annotated datasets and a user study, MIRA has demonstrated substantial improvements in the accuracy of instruction recommendation. The encouraging results highlight MIRA's potential to revolutionize the way users engage with AI services on their smartphones, offering a more seamless and efficient experience. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13773 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation Bian, Zhipeng Zhu, Jieming Xie, Xuyang Dai, Quanyu Zhao, Zhou Dong, Zhenhua Artificial Intelligence Information Retrieval I.2.7; I.2.10 The rapid advancement of generative AI technologies is driving the integration of diverse AI-powered services into smartphones, transforming how users interact with their devices. To simplify access to predefined AI services, this paper introduces MIRA, a pioneering framework for task instruction recommendation that enables intuitive one-touch AI tasking on smartphones. With MIRA, users can long-press on images or text objects to receive contextually relevant instruction recommendations for executing AI tasks. Our work introduces three key innovations: 1) A multimodal large language model (MLLM)-based recommendation pipeline with structured reasoning to extract key entities, infer user intent, and generate precise instructions; 2) A template-augmented reasoning mechanism that integrates high-level reasoning templates, enhancing task inference accuracy; 3) A prefix-tree-based constrained decoding strategy that restricts outputs to predefined instruction candidates, ensuring coherent and intent-aligned suggestions. Through evaluation using a real-world annotated datasets and a user study, MIRA has demonstrated substantial improvements in the accuracy of instruction recommendation. The encouraging results highlight MIRA's potential to revolutionize the way users engage with AI services on their smartphones, offering a more seamless and efficient experience. |
| title | MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation |
| topic | Artificial Intelligence Information Retrieval I.2.7; I.2.10 |
| url | https://arxiv.org/abs/2509.13773 |