Cover Image

Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ramrakhya, Ram, Chang, Matthew, Puig, Xavier, Desai, Ruta, Kira, Zsolt, Mottaghi, Roozbeh
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation
by: Khanna, Mukul, et al.
Published: (2024)

ADAPT: Actively Discovering and Adapting to Preferences for any Task
by: Patel, Maithili, et al.
Published: (2025)

PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
by: Chang, Matthew, et al.
Published: (2024)

Memo: Training Memory-Efficient Embodied Agents with Reinforcement Learning
by: Gupta, Gunshi, et al.
Published: (2025)

ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI
by: Elawady, Ahmad, et al.
Published: (2024)

User-in-the-loop Evaluation of Multimodal LLMs for Activity Assistance
by: Verghese, Mrinal, et al.
Published: (2024)

Hierarchical Reinforcement Learning and Value Optimization for Challenging Quadruped Locomotion
by: Coholich, Jeremiah, et al.
Published: (2025)

AgentAsk: Multi-Agent Systems Need to Ask
by: Lin, Bohan, et al.
Published: (2025)

HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?
by: Trinh, Tu, et al.
Published: (2026)

HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigation
by: Yokoyama, Naoki, et al.
Published: (2024)

ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
by: Chen, Hanyang, et al.
Published: (2025)

Reinforcement Learning via Auxiliary Task Distillation
by: Harish, Abhinav Narayan, et al.
Published: (2024)

Grounding LLMs in Scientific Discovery via Embodied Actions
by: Zhang, Bo, et al.
Published: (2026)

Scaling Synthetic Task Generation for Agents via Exploration
by: Ramrakhya, Ram, et al.
Published: (2025)

Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
by: Zheng, Xiang, et al.
Published: (2026)

ASH: Agents that Self-Hone via Embodied Learning
by: Schneider, Benjamin, et al.
Published: (2026)

Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework
by: Binkyte, Ruta
Published: (2025)

Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
by: Hu, Zican, et al.
Published: (2025)

Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
by: Fei, Zhaoye, et al.
Published: (2025)

Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks
by: Shojaeighadikolaei, Amin, et al.
Published: (2024)

Can LLMs Ask Good Questions?
by: Zhang, Yueheng, et al.
Published: (2025)

EMAC+: Embodied Multimodal Agent for Collaborative Planning with VLM+LLM
by: Ao, Shuang, et al.
Published: (2025)

Multi-Modal Grounded Planning and Efficient Replanning For Learning Embodied Agents with A Few Examples
by: Kim, Taewoong, et al.
Published: (2024)

Situated Instruction Following
by: Min, So Yeon, et al.
Published: (2024)

Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval
by: Hsu, Sheryl, et al.
Published: (2024)

Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
by: Li, Manling, et al.
Published: (2024)

Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents
by: Tan, Reuben, et al.
Published: (2025)

Align While Search: Belief-Guided Exploratory Inference for World-Grounded Embodied Agents
by: Bae, Seohui, et al.
Published: (2025)

RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
by: Gehring, Jonas, et al.
Published: (2024)

Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanation
by: Park, Jonghyuk, et al.
Published: (2024)

Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own
by: Ye, Weirui, et al.
Published: (2023)

Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments
by: Lu, Qingyu, et al.
Published: (2025)

Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning
by: Yuan, Xinbin, et al.
Published: (2025)

Grounding Agent Reasoning in Image Schemas: A Neurosymbolic Approach to Embodied Cognition
by: Olivier, François, et al.
Published: (2025)

Learning to Ask: When LLM Agents Meet Unclear Instruction
by: Wang, Wenxuan, et al.
Published: (2024)

Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification
by: Andrade, Moises, et al.
Published: (2025)

Continual Adaptation of Vision Transformers for Federated Learning
by: Halbe, Shaunak, et al.
Published: (2023)

IntelliAsk: Learning to Ask High-Quality Research Questions via RLVR
by: Sharma, Karun, et al.
Published: (2026)

Embodied Intelligence in Disassembly: Multimodal Perception Cross-validation and Continual Learning in Neuro-Symbolic TAMP
by: He, Ziwen, et al.
Published: (2025)

What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzles
by: Zhou, Mengtao, et al.
Published: (2025)