Cover Image

CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Myeongsoo, Garg, Shweta, Ray, Baishakhi, Kumar, Varun, Deoras, Anoop
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
by: Kim, Myeongsoo, et al.
Published: (2026)

CODESTRUCT: Code Agents over Structured Action Spaces
by: Kim, Myeongsoo, et al.
Published: (2026)

LeDex: Training LLMs to Better Self-Debug and Explain Code
by: Jiang, Nan, et al.
Published: (2024)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
by: Kuhar, Sachit, et al.
Published: (2024)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

CodeFort: Robust Training for Code Generation Models
by: Zhang, Yuhao, et al.
Published: (2024)

Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
by: Chen, Simin, et al.
Published: (2025)

UTFix: Change Aware Unit Test Repairing using LLM
by: Rahman, Shanto, et al.
Published: (2025)

MigrationBench: Repository-Level Code Migration Benchmark from Java 8
by: Liu, Linbo, et al.
Published: (2025)

SpecTra: Enhancing the Code Translation Ability of Language Models by Generating Multi-Modal Specifications
by: Nitin, Vikram, et al.
Published: (2024)

CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation
by: Wang, Sizhe, et al.
Published: (2025)

Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
by: Pujar, Saurabh, et al.
Published: (2025)

Structural Code Search using Natural Language Queries
by: Limpanukorn, Ben, et al.
Published: (2025)

Deep-Bench: Deep Learning Benchmark Dataset for Code Generation
by: Daghighfarsoodeh, Alireza, et al.
Published: (2025)

CodeSSM: Towards State Space Models for Code Understanding
by: Verma, Shweta, et al.
Published: (2025)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
by: Padwal, Vedant
Published: (2026)

A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs
by: Kim, Myeongsoo, et al.
Published: (2024)

SWE Context Bench: A Benchmark for Context Learning in Coding
by: Zhu, Jiayuan, et al.
Published: (2026)

FeatureBench: Benchmarking Agentic Coding for Complex Feature Development
by: Zhou, Qixing, et al.
Published: (2026)

LlamaRestTest: Effective REST API Testing with Small Language Models
by: Kim, Myeongsoo, et al.
Published: (2025)

EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
by: Zhang, Wentao, et al.
Published: (2026)

SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
by: Ding, Yangruibo, et al.
Published: (2024)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)

Code2Bench: Scaling Source and Rigor for Dynamic Benchmark Construction
by: Zhang, Zhe, et al.
Published: (2025)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
by: Yin, Wenjing, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
by: Ni, Ziyi, et al.
Published: (2025)

ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution
by: Wang, Yubang, et al.
Published: (2026)

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation
by: He, Yibo, et al.
Published: (2025)

SWE-QA: A Dataset and Benchmark for Complex Code Understanding
by: Elkoussy, Laïla, et al.
Published: (2026)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
by: Zhuo, Terry Yue, et al.
Published: (2024)

CYCLE: Learning to Self-Refine the Code Generation
by: Ding, Yangruibo, et al.
Published: (2024)

AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL
by: Stennett, Tyler, et al.
Published: (2025)

ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
by: Lu, Pengrui, et al.
Published: (2026)

DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation
by: Xiao, Jingyu, et al.
Published: (2025)

UA-Code-Bench: A Competitive Programming Benchmark for Evaluating LLM Code Generation in Ukrainian
by: Syromiatnikov, Mykyta, et al.
Published: (2025)

Code Quality Analysis of Translations from C to Rust
by: Tadesse, Biruk, et al.
Published: (2026)

Agentic AI Software Engineers: Programming with Trust
by: Roychoudhury, Abhik, et al.
Published: (2025)