Imagen de Portada

ClarEval: A Benchmark for Evaluating Clarification Skills of Code Agents under Ambiguous Instructions

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Jialin, Wu, Yuan, Chang, Yi
Formato:	Preprint
Publicado:	2026
Materias:	Software Engineering
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
por: Xie, Bang, et al.
Publicado: (2026)

HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
por: Wu, Jie JW, et al.
Publicado: (2024)

ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code
por: Feng, Jia, et al.
Publicado: (2024)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
por: Liu, Kaiyuan, et al.
Publicado: (2025)

SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
por: Ye, Zhifan, et al.
Publicado: (2025)

FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
por: Dai, Dekun, et al.
Publicado: (2025)

SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
por: Zhan, Zexun, et al.
Publicado: (2025)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
por: Paul, Debalina Ghosh, et al.
Publicado: (2024)

SolEval: Benchmarking Large Language Models for Repository-level Solidity Code Generation
por: Peng, Zhiyuan, et al.
Publicado: (2025)

ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation
por: Xue, Pengyu, et al.
Publicado: (2024)

CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
por: Yu, Hao, et al.
Publicado: (2023)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
por: Zhang, Tanghaoran, et al.
Publicado: (2026)

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
por: Li, Chenxin, et al.
Publicado: (2026)

Can Code Language Models Learn Clarification-Seeking Behaviors?
por: Wu, Jie JW, et al.
Publicado: (2025)

Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation
por: Jia, Haoxiang, et al.
Publicado: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
por: Li, Jia, et al.
Publicado: (2024)

SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications
por: Ma, Lezhi, et al.
Publicado: (2024)

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
por: Wu, Jiarong, et al.
Publicado: (2025)

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents
por: Hong, Sirui, et al.
Publicado: (2026)

ScratchEval : A Multimodal Evaluation Framework for LLMs in Block-Based Programming
por: Si, Yuan, et al.
Publicado: (2026)

SkillSmith: Compiling Agent Skills into Boundary-Guided Runtime Interfaces
por: Xu, Duling, et al.
Publicado: (2026)

ReleaseEval: A Benchmark for Evaluating Language Models in Automated Release Note Generation
por: Meng, Qianru, et al.
Publicado: (2025)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
por: Guo, Chengquan, et al.
Publicado: (2024)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
por: Kuhar, Sachit, et al.
Publicado: (2024)

A Benchmark for Evaluating Repository-Level Code Agents with Intermediate Reasoning on Feature Addition Task
por: Liu, Shuhan, et al.
Publicado: (2026)

StackEval: Benchmarking LLMs in Coding Assistance
por: Shah, Nidhish, et al.
Publicado: (2024)

EffiSkill: Agent Skill Based Automated Code Efficiency Optimization
por: Wang, Zimu, et al.
Publicado: (2026)

ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents
por: Jeon, YoungHoon, et al.
Publicado: (2026)

Scaling Coding Agents via Atomic Skills
por: Ma, Yingwei, et al.
Publicado: (2026)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
por: Wang, Jiexin, et al.
Publicado: (2024)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
por: Chen, Yeheng, et al.
Publicado: (2026)

When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions
por: Larbi, Maya, et al.
Publicado: (2025)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
por: Wu, Qinyun, et al.
Publicado: (2024)

EvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
por: Wen, Xin-Cheng, et al.
Publicado: (2024)

CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation
por: Yan, Kaiwen, et al.
Publicado: (2025)

Automatically Benchmarking LLM Code Agents through Agent-Driven Annotation and Evaluation
por: Fu, Lingyue, et al.
Publicado: (2025)

Is Your Benchmark (Still) Useful? Dynamic Benchmarking for Code Language Models
por: Guan, Batu, et al.
Publicado: (2025)

A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedback
por: Duan, Guoliang, et al.
Publicado: (2025)

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
por: Lu, Qinghua, et al.
Publicado: (2025)

CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
por: Zhang, Qingyu, et al.
Publicado: (2025)