Copertina

HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devices

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Silin, Guo, Yuhang, Yao, Jiashu, Liu, Zeming, Wang, Haifeng
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

HomeSafeBench: A Benchmark for Embodied Vision-Language Models in Free-Exploration Home Safety Inspection
di: Gao, Siyuan, et al.
Pubblicazione: (2025)

PersonalHomeBench: Evaluating Agents in Personalized Smart Homes
di: Bharadwaj, Manasa, et al.
Pubblicazione: (2026)

Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation
di: Yao, Jiashu, et al.
Pubblicazione: (2026)

Deterministic Reversible Data Augmentation for Neural Machine Translation
di: Yao, Jiashu, et al.
Pubblicazione: (2024)

FAME: Towards Factual Multi-Task Model Editing
di: Zeng, Li, et al.
Pubblicazione: (2024)

ReFF: Reinforcing Format Faithfulness in Language Models across Varied Tasks
di: Yao, Jiashu, et al.
Pubblicazione: (2024)

On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation
di: Birkmose, Rune, et al.
Pubblicazione: (2025)

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
di: Yao, Jiashu, et al.
Pubblicazione: (2026)

SimuHome: A Temporal- and Environment-Aware Benchmark for Smart Home LLM Agents
di: Seo, Gyuhyeon, et al.
Pubblicazione: (2025)

Optimizing Chain-of-Thought Reasoning: Tackling Arranging Bottleneck via Plan Augmentation
di: Qiu, Yuli, et al.
Pubblicazione: (2024)

ToolSpectrum : Towards Personalized Tool Utilization for Large Language Models
di: Cheng, Zihao, et al.
Pubblicazione: (2025)

SmartBench: Evaluating LLMs in Smart Homes with Anomalous Device States and Behavioral Contexts
di: Zou, Qingsong, et al.
Pubblicazione: (2026)

Leveraging Large Language Models for Explainable Activity Recognition in Smart Homes: A Critical Evaluation
di: Fiori, Michele, et al.
Pubblicazione: (2025)

Exploring In-Image Machine Translation with Real-World Background
di: Tian, Yanzhi, et al.
Pubblicazione: (2025)

POLIS-Bench: Towards Multi-Dimensional Evaluation of LLMs for Bilingual Policy Tasks in Governmental Scenarios
di: Yang, Tingyue, et al.
Pubblicazione: (2025)

MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs
di: Qian, Yusu, et al.
Pubblicazione: (2024)

MiCU: End-to-End Smart Home Command Understanding with Large Language Model
di: Han, Haowei, et al.
Pubblicazione: (2026)

CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages
di: Yang, Yilun, et al.
Pubblicazione: (2025)

Evaluating a Multi-Agent Voice-Enabled Smart Speaker for Care Homes: A Safety-Focused Framework
di: Dehghani, Zeinab, et al.
Pubblicazione: (2026)

SmartHome-Bench: A Comprehensive Benchmark for Video Anomaly Detection in Smart Homes Using Multi-Modal Large Language Models
di: Zhao, Xinyi, et al.
Pubblicazione: (2025)

SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs
di: Xia, Hongfei, et al.
Pubblicazione: (2025)

Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation
di: Tian, Yanzhi, et al.
Pubblicazione: (2026)

DocMEdit: Towards Document-Level Model Editing
di: Zeng, Li, et al.
Pubblicazione: (2025)

InFoBench: Evaluating Instruction Following Ability in Large Language Models
di: Qin, Yiwei, et al.
Pubblicazione: (2024)

IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis
di: Li, Hanyu, et al.
Pubblicazione: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
di: Wen, Bosi, et al.
Pubblicazione: (2026)

NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities
di: Li, Mo, et al.
Pubblicazione: (2024)

Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges
di: Shi, Xiaoming, et al.
Pubblicazione: (2024)

ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
di: Li, Bohan, et al.
Pubblicazione: (2025)

MolViBench: Evaluating LLMs on Molecular Vibe Coding
di: Li, Jiatong, et al.
Pubblicazione: (2026)

A General-Purpose Device for Interaction with LLMs
di: Xu, Jiajun, et al.
Pubblicazione: (2024)

DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science
di: Shu, Fan, et al.
Pubblicazione: (2026)

RuozhiBench: Evaluating LLMs with Logical Fallacies and Misleading Premises
di: Zhai, Zenan, et al.
Pubblicazione: (2025)

Towards Position-Robust Talent Recommendation via Large Language Models
di: Du, Silin, et al.
Pubblicazione: (2026)

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
di: Ge, Wentao, et al.
Pubblicazione: (2023)

Layout Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)
di: Thukral, Megha, et al.
Pubblicazione: (2024)

Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models
di: Huang, Yupan, et al.
Pubblicazione: (2023)

Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Game
di: Du, Silin, et al.
Pubblicazione: (2024)

Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcement
di: Yao, Jiashu, et al.
Pubblicazione: (2025)

PRIM: Towards Practical In-Image Multilingual Machine Translation
di: Tian, Yanzhi, et al.
Pubblicazione: (2025)