Cover Image

Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Simin, Pusarla, Pranav, Ray, Baishakhi
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
by: Ding, Yangruibo, et al.
Published: (2024)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

A Code Comprehension Benchmark for Large Language Models for Code
by: Havare, Jayant, et al.
Published: (2025)

PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models
by: Chen, Simin, et al.
Published: (2024)

CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale
by: Wang, Chenlong, et al.
Published: (2025)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
by: Guo, Jiawei, et al.
Published: (2024)

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2024)

LeDex: Training LLMs to Better Self-Debug and Explain Code
by: Jiang, Nan, et al.
Published: (2024)

Eliciting Instruction-tuned Code Language Models' Capabilities to Utilize Auxiliary Function for Code Generation
by: Lee, Seonghyeon, et al.
Published: (2024)

Exploring Data-Efficient Adaptation of Large Language Models for Code Generation
by: Jiang, Xue, et al.
Published: (2024)

SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning
by: Peng, Jinjun, et al.
Published: (2026)

DebugBench: Evaluating Debugging Capability of Large Language Models
by: Tian, Runchu, et al.
Published: (2024)

CodeMirage: Hallucinations in Code Generated by Large Language Models
by: Agarwal, Vibhor, et al.
Published: (2024)

WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models
by: Fan, Shengda, et al.
Published: (2024)

CODEMENV: Benchmarking Large Language Models on Code Migration
by: Cheng, Keyuan, et al.
Published: (2025)

Self-Explained Keywords Empower Large Language Models for Code Generation
by: Fan, Lishui, et al.
Published: (2024)

LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Models
by: Feng, Xiaoning, et al.
Published: (2022)

Testing the Effect of Code Documentation on Large Language Model Code Understanding
by: Macke, William, et al.
Published: (2024)

CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
by: Kim, Myeongsoo, et al.
Published: (2025)

Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy
by: Dong, Yihong, et al.
Published: (2026)

Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code
by: Jiang, Nan, et al.
Published: (2024)

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages
by: Kammakomati, Mehant, et al.
Published: (2024)

ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models
by: Han, Hojae, et al.
Published: (2024)

A Survey on Large Language Models for Code Generation
by: Jiang, Juyong, et al.
Published: (2024)

Analyzing the Performance of Large Language Models on Code Summarization
by: Haldar, Rajarshi, et al.
Published: (2024)

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency
by: Huang, Baizhou, et al.
Published: (2023)

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
by: Adamenko, Pavel, et al.
Published: (2025)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

ICE-Score: Instructing Large Language Models to Evaluate Code
by: Zhuo, Terry Yue
Published: (2023)

Code Readability in the Age of Large Language Models: An Industrial Case Study from Atlassian
by: Takerngsaksiri, Wannita, et al.
Published: (2025)

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases
by: Liu, Xiangyan, et al.
Published: (2024)

Mitigating Gender Bias in Code Large Language Models via Model Editing
by: Qin, Zhanyue, et al.
Published: (2024)

Large Language Model Critics for Execution-Free Evaluation of Code Changes
by: Yadavally, Aashish, et al.
Published: (2025)

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
by: Zhuo, Terry Yue, et al.
Published: (2024)

E2Edev: Benchmarking Large Language Models in End-to-End Software Development Task
by: Liu, Jingyao, et al.
Published: (2025)

Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
by: Wang, Hanbin, et al.
Published: (2025)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
by: Yang, Jialin, et al.
Published: (2025)

Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets
by: Jelodar, Hamed, et al.
Published: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)