PersonalHomeBench: Evaluating Agents in Personalized Smart Homes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914564003594240 |
|---|---|
| author | Bharadwaj, Manasa Liu, Yolanda Yang, InJung Kim, Sungil Verma, Nikhil Kim, KoKeun Ferreira, Kevin Kim, YoungJoon |
| author_facet | Bharadwaj, Manasa Liu, Yolanda Yang, InJung Kim, Sungil Verma, Nikhil Kim, KoKeun Ferreira, Kevin Kim, YoungJoon |
| contents | Agentic AI systems are rapidly advancing toward real-world applications, yet their readiness in complex and personalized environments remains insufficiently characterized. To address this gap, we introduce PersonalHomeBench, a benchmark for evaluating foundation models as agentic assistants in personalized smart home environments. The benchmark is constructed through an iterative process that progressively builds rich household states, which are then used to generate personalized, context-dependent tasks. To support realistic agent-environment interaction, we provide PersonalHomeTools, a comprehensive toolbox enabling household information retrieval, appliance control, and situational understanding. PersonalHomeBench evaluates both reactive and proactive agentic abilities under unimodal and multimodal observations. Thorough experimentation reveals a systematic performance reduction as task complexity increases, with pronounced failures in counterfactual reasoning and under partial observability, where effective tool-based information gathering is required. These results position PersonalHomeBench as a rigorous evaluation platform for analyzing the robustness and limitations of personalized agentic reasoning and planning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16813 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PersonalHomeBench: Evaluating Agents in Personalized Smart Homes Bharadwaj, Manasa Liu, Yolanda Yang, InJung Kim, Sungil Verma, Nikhil Kim, KoKeun Ferreira, Kevin Kim, YoungJoon Artificial Intelligence Computation and Language Databases Agentic AI systems are rapidly advancing toward real-world applications, yet their readiness in complex and personalized environments remains insufficiently characterized. To address this gap, we introduce PersonalHomeBench, a benchmark for evaluating foundation models as agentic assistants in personalized smart home environments. The benchmark is constructed through an iterative process that progressively builds rich household states, which are then used to generate personalized, context-dependent tasks. To support realistic agent-environment interaction, we provide PersonalHomeTools, a comprehensive toolbox enabling household information retrieval, appliance control, and situational understanding. PersonalHomeBench evaluates both reactive and proactive agentic abilities under unimodal and multimodal observations. Thorough experimentation reveals a systematic performance reduction as task complexity increases, with pronounced failures in counterfactual reasoning and under partial observability, where effective tool-based information gathering is required. These results position PersonalHomeBench as a rigorous evaluation platform for analyzing the robustness and limitations of personalized agentic reasoning and planning. |
| title | PersonalHomeBench: Evaluating Agents in Personalized Smart Homes |
| topic | Artificial Intelligence Computation and Language Databases |
| url | https://arxiv.org/abs/2604.16813 |