Image de couverture de livre

Evaluating LLMs on Sequential API Call Through Automated Test Generation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Huang, Yuheng, Song, Jiayang, Song, Da, Ji, Zhenlan, Wang, Wenhan, Wang, Shuai, Ma, Lei
Format:	Preprint
Publié:	2025
Sujets:	Software Engineering Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
par: Sun, Ruoyu, et autres
Publié: (2025)

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
par: Xie, Xuan, et autres
Publié: (2024)

Digging Into the Internal: Causality-Based Analysis of LLM Function Calling
par: Ji, Zhenlan, et autres
Publié: (2025)

LeCov: Multi-level Testing Criteria for Large Language Models
par: Xie, Xuan, et autres
Publié: (2024)

Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
par: Huang, Yuheng, et autres
Publié: (2023)

LUNA: A Model-Based Universal Analysis Framework for Large Language Models
par: Song, Da, et autres
Publié: (2023)

AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling
par: Huang, Yuheng, et autres
Publié: (2024)

Evaluating Retrieval-Augmented Generation Variants for Natural Language-Based SQL and API Call Generation
par: Marketsmüller, Michael, et autres
Publié: (2026)

CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems
par: Lyu, Zongyi, et autres
Publié: (2026)

Reverse Chain: A Generic-Rule for LLMs to Master Multi-API Planning
par: Zhang, Yinger, et autres
Publié: (2023)

Risk Assessment Framework for Code LLMs via Leveraging Internal States
par: Huang, Yuheng, et autres
Publié: (2025)

VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
par: Wang, Zhijie, et autres
Publié: (2024)

TESTEVAL: Benchmarking Large Language Models for Test Case Generation
par: Wang, Wenhan, et autres
Publié: (2024)

Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
par: Zhao, Shengming, et autres
Publié: (2024)

Compositional API Recommendation for Library-Oriented Code Generation
par: Ma, Zexiong, et autres
Publié: (2024)

Evaluating Implicit Regulatory Compliance in LLM Tool Invocation via Logic-Guided Synthesis
par: Song, Da, et autres
Publié: (2026)

ToolFactory: Automating Tool Generation by Leveraging LLM to Understand REST API Documentations
par: Ni, Xinyi, et autres
Publié: (2025)

AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL
par: Stennett, Tyler, et autres
Publié: (2025)

Fine-grained Testing for Autonomous Driving Software: a Study on Autoware with LLM-driven Unit Testing
par: Wang, Wenhan, et autres
Publié: (2025)

Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
par: Wang, Hanbin, et autres
Publié: (2025)

Assessing Evaluation Metrics for Neural Test Oracle Generation
par: Shin, Jiho, et autres
Publié: (2023)

Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation
par: Lyu, Zongyi, et autres
Publié: (2025)

The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
par: Gao, Shuzheng, et autres
Publié: (2025)

Live API-Bench: 2500+ Live APIs for Testing Multi-Step Tool Calling
par: Elder, Benjamin, et autres
Publié: (2025)

APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
par: Liu, Zuxin, et autres
Publié: (2024)

Can LLMs Generate Reliable Test Case Generators? A Study on Competition-Level Programming Problems
par: Cao, Yuhan, et autres
Publié: (2025)

Automating a Complete Software Test Process Using LLMs: An Automotive Case Study
par: Wang, Shuai, et autres
Publié: (2025)

Towards Automated Smart Contract Generation: Evaluation, Benchmarking, and Retrieval-Augmented Repair
par: Chen, Zaoyu, et autres
Publié: (2025)

SoAy: A Solution-based LLM API-using Methodology for Academic Information Seeking
par: Wang, Yuanchun, et autres
Publié: (2024)

Python Symbolic Execution with LLM-powered Code Generation
par: Wang, Wenhan, et autres
Publié: (2024)

Revolutionizing API Documentation through Summarization
par: Naghshzan, AmirHossein, et autres
Publié: (2024)

MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems
par: Wang, Renzhi, et autres
Publié: (2024)

Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
par: Chen, Kexin, et autres
Publié: (2024)

ACECODER: Acing Coder RL via Automated Test-Case Synthesis
par: Zeng, Huaye, et autres
Publié: (2025)

Combining TSL and LLM to Automate REST API Testing: A Comparative Study
par: Barradas, Thiago, et autres
Publié: (2025)

Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
par: Wang, Wenxuan
Publié: (2024)

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
par: Yan, Weixiang, et autres
Publié: (2023)

Are LLMs Correctly Integrated into Software Systems?
par: Shao, Yuchen, et autres
Publié: (2024)

Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
par: Wang, Zhijie, et autres
Publié: (2024)

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency
par: Huang, Baizhou, et autres
Publié: (2023)