Cover Image

SmartBench: Evaluating LLMs in Smart Homes with Anomalous Device States and Behavioral Contexts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zou, Qingsong, Yan, Zhi, Xu, Zhiyao, Gao, Kuofeng, Xiao, Jingyu, Jiang, Yong
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Semantic-aware Graph-guided Behavior Sequences Generation with Large Language Models for Smart Homes
by: Xu, Zhiyao, et al.
Published: (2025)

Synthetic User Behavior Sequence Generation with Large Language Models for Smart Homes
by: Xu, Zhiyao, et al.
Published: (2025)

Make Your Home Safe: Time-aware Unsupervised User Behavior Anomaly Detection in Smart Homes via Loss-guided Mask
by: Xiao, Jingyu, et al.
Published: (2024)

PersonalHomeBench: Evaluating Agents in Personalized Smart Homes
by: Bharadwaj, Manasa, et al.
Published: (2026)

QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language
by: Zou, Qingsong, et al.
Published: (2025)

SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
by: Li, Kuan, et al.
Published: (2026)

Trust Your Memory: Verifiable Control of Smart Homes through Reinforcement Learning with Multi-dimensional Rewards
by: Guo, Kai-Yuan, et al.
Published: (2026)

SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant?
by: Lu, Xudong, et al.
Published: (2025)

A Systematic Review of Security Vulnerabilities in Smart Home Devices and Mitigation Techniques
by: Alzaylaee, Mohammed K.
Published: (2025)

SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
by: Chen, Jingxuan, et al.
Published: (2024)

MiCU: End-to-End Smart Home Command Understanding with Large Language Model
by: Han, Haowei, et al.
Published: (2026)

Empirical evaluation of LLMs in predicting fixes of Configuration bugs in Smart Home System
by: Monisha, Sheikh Moonwara Anjum, et al.
Published: (2025)

HomeFlow: A Data Flywheel for Smart Home Agent Training with Verifiable Simulation
by: Gu, Yi, et al.
Published: (2026)

Guiding LLM-based Smart Contract Generation with Finite State Machine
by: Luo, Hao, et al.
Published: (2025)

Identifying and Addressing User-level Security Concerns in Smart Homes Using "Smaller" LLMs
by: Chowdhury, Hafijul Hoque, et al.
Published: (2025)

Edge-FIT: Federated Instruction Tuning of Quantized LLMs for Privacy-Preserving Smart Home Environments
by: Venkatesh, Vinay, et al.
Published: (2025)

DomusFM: A Foundation Model for Smart-Home Sensor Data
by: Fiori, Michele, et al.
Published: (2026)

SimuHome: A Temporal- and Environment-Aware Benchmark for Smart Home LLM Agents
by: Seo, Gyuhyeon, et al.
Published: (2025)

Teaching Machines to Code: Smart Contract Translation with LLMs
by: Karanjai, Rabimba, et al.
Published: (2024)

Logic Meets Magic: LLMs Cracking Smart Contract Vulnerabilities
by: Xiao, ZeKe, et al.
Published: (2025)

Intelligence of Things: A Spatial Context-Aware Control System for Smart Devices
by: Kalivarathan, Sukanth, et al.
Published: (2025)

Balancing Usability and Compliance in AI Smart Devices: A Privacy-by-Design Audit of Google Home, Alexa, and Siri
by: De Clark, Trevor, et al.
Published: (2026)

DevPiolt: Operation Recommendation for IoT Devices at Xiaomi Home
by: Wang, Yuxiang, et al.
Published: (2025)

Towards Secure Program Partitioning for Smart Contracts with LLM's In-Context Learning
by: Liu, Ye, et al.
Published: (2025)

Evaluating a Multi-Agent Voice-Enabled Smart Speaker for Care Homes: A Safety-Focused Framework
by: Dehghani, Zeinab, et al.
Published: (2026)

HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devices
by: Li, Silin, et al.
Published: (2025)

SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber World
by: Zhang, Jiaqi, et al.
Published: (2024)

SC-Bench: A Large-Scale Dataset for Smart Contract Auditing
by: Xia, Shihao, et al.
Published: (2024)

Timing Matters: Enhancing User Experience through Temporal Prediction in Smart Homes
by: Ganatra, Shrey, et al.
Published: (2024)

MoralBench: Moral Evaluation of LLMs
by: Ji, Jianchao, et al.
Published: (2024)

LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
by: Rando, Stefano, et al.
Published: (2025)

Predicting Fine-grained Behavioral and Psychological Symptoms of Dementia Based on Machine Learning and Smart Wearable Devices
by: Hsu, Benny Wei-Yun, et al.
Published: (2024)

MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?
by: Zou, Xingze, et al.
Published: (2026)

SmartPlay: A Benchmark for LLMs as Intelligent Agents
by: Wu, Yue, et al.
Published: (2023)

Leveraging Large Language Models for enhanced personalised user experience in Smart Homes
by: Rey-Jouanchicot, Jordan, et al.
Published: (2024)

Sasha: Creative Goal-Oriented Reasoning in Smart Homes with Large Language Models
by: King, Evan, et al.
Published: (2023)

The Role of LLMs in Sustainable Smart Cities: Applications, Challenges, and Future Directions
by: Ullah, Amin, et al.
Published: (2024)

Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models
by: Huang, Xinyu, et al.
Published: (2025)

GNN-XAR: A Graph Neural Network for Explainable Activity Recognition in Smart Homes
by: Fiori, Michele, et al.
Published: (2025)

HearthNet: Edge Multi-Agent Orchestration for Smart Homes
by: Zhan, Zhonghao, et al.
Published: (2026)