Cover Image

HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Device Scenarios

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jun, Zhou, Jiamu, Wen, Muning, Mo, Xiaoyun, Zhang, Haoyu, Lin, Qiqiang, Jin, Cheng, Wang, Xihuai, Zhang, Weinan, Peng, Qiuying
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
by: Lin, Qiqiang, et al.
Published: (2024)

P3: A Policy-Driven, Pace-Adaptive, and Diversity-Promoted Framework for data pruning in LLM Training
by: Yang, Yingxuan, et al.
Published: (2024)

MobileUse: A GUI Agent with Hierarchical Reflection for Autonomous Mobile Operation
by: Li, Ning, et al.
Published: (2025)

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm
by: Song, Chao, et al.
Published: (2024)

ColorBench: Benchmarking Mobile Agents with Graph-Structured Framework for Complex Long-Horizon Tasks
by: Song, Yuanyi, et al.
Published: (2025)

MARFT: Multi-Agent Reinforcement Fine-Tuning
by: Liao, Junwei, et al.
Published: (2025)

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios
by: Wang, Sizhe, et al.
Published: (2026)

RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
by: Zhang, Jun, et al.
Published: (2025)

Mutual Theory of Mind in Human-AI Collaboration: An Empirical Study with LLM-driven AI Agents in a Real-time Shared Workspace Task
by: Zhang, Shao, et al.
Published: (2024)

ColorAgent: Building A Robust, Personalized, and Interactive OS Agent
by: Li, Ning, et al.
Published: (2025)

PARL-MT: Learning to Call Functions in Multi-Turn Conversation with Progress Awareness
by: Chai, Huacan, et al.
Published: (2025)

Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
by: Feng, Xidong, et al.
Published: (2023)

Computing Ex Ante Equilibrium in Heterogeneous Zero-Sum Team Games
by: Liu, Naming, et al.
Published: (2024)

CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks
by: Fu, Lingyue, et al.
Published: (2025)

Adaptive Milestone Reward for GUI Agents
by: Zheng, Congmin, et al.
Published: (2026)

ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination
by: Wang, Xihuai, et al.
Published: (2023)

Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration
by: Zhang, Shao, et al.
Published: (2025)

ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario
by: Zhong, Lucen, et al.
Published: (2025)

ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution
by: Wang, Jihong, et al.
Published: (2026)

Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects
by: Wang, Xihuai, et al.
Published: (2022)

Reinforcing Language Agents via Policy Optimization with Action Decomposition
by: Wen, Muning, et al.
Published: (2024)

Chain of Functions: A Programmatic Pipeline for Fine-Grained Chart Reasoning Data
by: Li, Zijian, et al.
Published: (2025)

Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline
by: Zeng, Guancheng, et al.
Published: (2024)

Position: Agentic AI System Is a Foreseeable Pathway to AGI
by: Liao, Junwei, et al.
Published: (2026)

PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs
by: Peng, Dan, et al.
Published: (2024)

PS$^2$: Parameterized Control for Fine-Grained Student Proficiency Simulation
by: Liu, Ruochen, et al.
Published: (2026)

Invertible Coarse Graining with Physics-Informed Generative Artificial Intelligence
by: Zhang, Jun, et al.
Published: (2023)

Towards Real-time Video Compressive Sensing on Mobile Devices
by: Cao, Miao, et al.
Published: (2024)

Mobile-GS: Real-time Gaussian Splatting for Mobile Devices
by: Du, Xiaobiao, et al.
Published: (2026)

TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision
by: Zhou, Ruiwen, et al.
Published: (2024)

ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning
by: Huang, Shulin, et al.
Published: (2025)

Distributed Real-Time Vehicle Control for Emergency Vehicle Transit: A Scalable Cooperative Method
by: Wang, WenXi, et al.
Published: (2026)

RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios
by: Zhao, Fei, et al.
Published: (2025)

STaR-SQL: Self-Taught Reasoner for Text-to-SQL
by: He, Mingqian, et al.
Published: (2025)

Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis
by: Zheng, Yujie, et al.
Published: (2026)

Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
by: Wen, Muning, et al.
Published: (2024)

LLM-based Multi-Agent Systems: Techniques and Business Perspectives
by: Yang, Yingxuan, et al.
Published: (2024)

S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Framework
by: Li, Yingshu, et al.
Published: (2025)

Quick on the Uptake: Eliciting Implicit Intents from Human Demonstrations for Personalized Mobile-Use Agents
by: Wu, Zheng, et al.
Published: (2025)

3M-TI: High-Quality Mobile Thermal Imaging via Calibration-free Multi-Camera Cross-Modal Diffusion
by: Chen, Minchong, et al.
Published: (2025)