Cover Image

CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Shiting, Fang, Zhen, Chen, Zehui, Yuan, Siyu, Ye, Junjie, Zeng, Yu, Chen, Lin, Mao, Qi, Zhao, Feng
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
by: Su, Qisheng, et al.
Published: (2026)

TestBench: Evaluating Class-Level Test Case Generation Capability of Large Language Models
by: Zhang, Quanjun, et al.
Published: (2024)

Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline
by: Zeng, Guancheng, et al.
Published: (2024)

SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineering
by: Ren, Qingnan, et al.
Published: (2026)

Evaluating the Capability of LLMs in Identifying Compilation Errors in Configurable Systems
by: Albuquerque, Lucas, et al.
Published: (2024)

Reflective Unit Test Generation for Precise Type Error Detection with Large Language Models
by: Yang, Chen, et al.
Published: (2025)

Hierarchical Evaluation of Software Design Capabilities of Large Language Models of Code
by: Saad, Mootez, et al.
Published: (2025)

ScaleCall -- Agentic Tool Calling at Scale for Fintech: Challenges, Methods, and Deployment Insights
by: Osuagwu, Richard, et al.
Published: (2025)

A Tool for Test Case Scenarios Generation Using Large Language Models
by: Sami, Abdul Malik, et al.
Published: (2024)

A Lightweight Framework for Adaptive Retrieval In Code Completion With Critique Model
by: Zhang, Wenrui, et al.
Published: (2024)

Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
by: Lu, Yuxuan, et al.
Published: (2026)

From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMapps
by: Mao, Yuetian, et al.
Published: (2025)

Aligning Requirement for Large Language Model's Code Generation
by: Tian, Zhao, et al.
Published: (2025)

WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality
by: Li, Chunyang, et al.
Published: (2025)

Improving Compiler Bug Isolation by Leveraging Large Language Models
by: Qi, Yixian, et al.
Published: (2025)

A Tool for Benchmarking Large Language Models' Robustness in Assessing the Realism of Driving Scenarios
by: Wu, Jiahui, et al.
Published: (2025)

Online-Optimized RAG for Tool Use and Function Calling
by: Pan, Yu, et al.
Published: (2025)

On the Evaluation of Large Language Models in Multilingual Vulnerability Repair
by: wang, Dong, et al.
Published: (2025)

A Survey on Evaluating Large Language Models in Code Generation Tasks
by: Chen, Liguo, et al.
Published: (2024)

AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarios
by: Shi, Kou, et al.
Published: (2026)

ASA: Training-Free Representation Engineering for Tool-Calling Agents
by: Wang, Youjin, et al.
Published: (2026)

Semantic-Enhanced Indirect Call Analysis with Large Language Models
by: Cheng, Baijun, et al.
Published: (2024)

Repairing Tool Calls Using Post-tool Execution Reflection and RAG
by: Tsay, Jason, et al.
Published: (2025)

Fixing Large Language Models' Specification Misunderstanding for Better Code Generation
by: Tian, Zhao, et al.
Published: (2023)

Evaluating Large Language Models for Multilingual Vulnerability Detection at Dual Granularities
by: Shu, Honglin, et al.
Published: (2025)

On the Evaluation of Large Language Models in Unit Test Generation
by: Yang, Lin, et al.
Published: (2024)

Scenario-Guided LLM-based Mobile App GUI Testing
by: Yu, Shengcheng, et al.
Published: (2025)

Practical, Automated Scenario-based Mobile App Testing
by: Yu, Shengcheng, et al.
Published: (2024)

Towards Automated Driving Violation Cause Analysis in Scenario-Based Testing for Autonomous Driving Systems
by: Wan, Ziwen, et al.
Published: (2024)

Clawdrain: Exploiting Tool-Calling Chains for Stealthy Token Exhaustion in OpenClaw Agents
by: Dong, Ben, et al.
Published: (2026)

A Comprehensive Study on Static Application Security Testing (SAST) Tools for Android
by: Zhu, Jingyun, et al.
Published: (2024)

Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation
by: Yang, Chen, et al.
Published: (2026)

Advancing Code Coverage: Incorporating Program Analysis with Large Language Models
by: Yang, Chen, et al.
Published: (2024)

Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction
by: Wu, Yifan, et al.
Published: (2024)

Development of a Evaluation Tool for Age-Appropriate Software in Aging Environments: A Delphi Study
by: Bai, Zhenggang, et al.
Published: (2024)

Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety
by: Chen, Xuan, et al.
Published: (2026)

SoVAR: Building Generalizable Scenarios from Accident Reports for Autonomous Driving Testing
by: Guo, An, et al.
Published: (2024)

A Large-scale Empirical Study on Fine-tuning Large Language Models for Unit Testing
by: Shang, Ye, et al.
Published: (2024)

The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
by: Xu, Haoyuan, et al.
Published: (2026)

Automatic Smart Contract Comment Generation via Large Language Models and In-Context Learning
by: Zhao, Junjie, et al.
Published: (2023)