Cover Image

Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistant

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Lei, Shen, Xiaoyu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SLURP-TN : Resource for Tunisian Dialect Spoken Language Understanding
by: Elleuch, Haroun, et al.
Published: (2026)

Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis
by: Mok, Jisoo, et al.
Published: (2025)

GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant
by: Shen, Zhuokang, et al.
Published: (2026)

PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants
by: Zhao, Zheng, et al.
Published: (2025)

AutoJudger: An Agent-Driven Framework for Efficient Benchmarking of MLLMs
by: Ding, Xuanwen, et al.
Published: (2025)

Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues
by: Sun, Lei, et al.
Published: (2024)

PersonalHomeBench: Evaluating Agents in Personalized Smart Homes
by: Bharadwaj, Manasa, et al.
Published: (2026)

MultiVox: A Benchmark for Evaluating Voice Assistants for Multimodal Interactions
by: Selvakumar, Ramaneswaran, et al.
Published: (2025)

AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
by: Oh, Gyutaek, et al.
Published: (2025)

DITING: A Multi-Agent Evaluation Framework for Benchmarking Web Novel Translation
by: Zhang, Enze, et al.
Published: (2025)

Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
by: Wang, Siyuan, et al.
Published: (2024)

PTCBENCH: Benchmarking Contextual Stability of Personality Traits in LLM Systems
by: Yu, Jiongchi, et al.
Published: (2026)

ES-MemEval: Benchmarking Conversational Agents on Personalized Long-Term Emotional Support
by: Chen, Tiantian, et al.
Published: (2026)

TravelAgent: An AI Assistant for Personalized Travel Planning
by: Chen, Aili, et al.
Published: (2024)

Aligning VLM Assistants with Personalized Situated Cognition
by: Li, Yongqi, et al.
Published: (2025)

How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants
by: Feng, Xueyang, et al.
Published: (2026)

Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines
by: Zhu, Yaochen, et al.
Published: (2023)

MultiJustice: A Chinese Dataset for Multi-Party, Multi-Charge Legal Prediction
by: Wang, Xiao, et al.
Published: (2025)

MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration
by: Zhou, Yucheng, et al.
Published: (2025)

Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration
by: Wang, Junyang, et al.
Published: (2024)

Hybrid OCR-LLM Framework for Enterprise-Scale Document Information Extraction Under Copy-heavy Task
by: Wang, Zilong, et al.
Published: (2025)

AutoPatent: A Multi-Agent Framework for Automatic Patent Generation
by: Wang, Qiyao, et al.
Published: (2024)

AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework
by: Fan, Meihao, et al.
Published: (2024)

AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML
by: Trirat, Patara, et al.
Published: (2024)

SALAD: Smart AI Language Assistant Daily
by: Nihal, Ragib Amin, et al.
Published: (2024)

CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model
by: Lei, Yang, et al.
Published: (2023)

LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation
by: Duan, Feiyu, et al.
Published: (2026)

AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions
by: Li, Ziming, et al.
Published: (2024)

Large Language Models Empowered Personalized Web Agents
by: Cai, Hongru, et al.
Published: (2024)

GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning
by: Cheng, Xiang, et al.
Published: (2026)

Talking to Data: Designing Smart Assistants for Humanities Databases
by: Sergeev, Alexander, et al.
Published: (2025)

Evaluating a Multi-Agent Voice-Enabled Smart Speaker for Care Homes: A Safety-Focused Framework
by: Dehghani, Zeinab, et al.
Published: (2026)

PRISM: A Personality-Driven Multi-Agent Framework for Social Media Simulation
by: Lu, Zhixiang, et al.
Published: (2025)

EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences
by: Shen, Jocelyn, et al.
Published: (2024)

Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent
by: Li, Xingzuo, et al.
Published: (2025)

YIELD: A Large-Scale Dataset and Evaluation Framework for Information Elicitation Agents
by: De Lima, Victor, et al.
Published: (2026)

WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents
by: Peeters, Ralph, et al.
Published: (2025)

A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support
by: Nguyen, Long S. T., et al.
Published: (2025)

FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
by: Luo, Junyu, et al.
Published: (2025)

JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework
by: Liu, Fan, et al.
Published: (2024)