Cover Image

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Roy, Monoshi Kumar, Chen, Simin, Steenhoek, Benjamin, Peng, Jinjun, Kaiser, Gail, Ray, Baishakhi, Le, Wei
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
by: Ding, Yangruibo, et al.
Published: (2024)

Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
by: Chen, Simin, et al.
Published: (2025)

CYCLE: Learning to Self-Refine the Code Generation
by: Ding, Yangruibo, et al.
Published: (2024)

Code Reasoning for Software Engineering Tasks: A Survey and A Call to Action
by: Pujar, Saurabh, et al.
Published: (2025)

CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
by: Kim, Myeongsoo, et al.
Published: (2025)

To Err is Machine: Vulnerability Detection Challenges LLM Reasoning
by: Steenhoek, Benjamin, et al.
Published: (2024)

CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generation
by: Peng, Jinjun, et al.
Published: (2025)

Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers
by: Chen, Simin, et al.
Published: (2025)

SpecTra: Enhancing the Code Translation Ability of Language Models by Generating Multi-Modal Specifications
by: Nitin, Vikram, et al.
Published: (2024)

REINFOREST: Reinforcing Semantic Code Similarity for Cross-Lingual Code Search Models
by: Saieva, Anthony, et al.
Published: (2023)

Trustworthy AI Software Engineers
by: Aleti, Aldeida, et al.
Published: (2026)

Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities
by: Chen, Simin, et al.
Published: (2025)

Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation
by: Garg, Spandan, et al.
Published: (2025)

Code Quality Analysis of Translations from C to Rust
by: Tadesse, Biruk, et al.
Published: (2026)

Evaluating LLMs Code Reasoning Under Real-World Context
by: Liu, Changshu
Published: (2026)

Automated Code Editing with Search-Generate-Modify
by: Liu, Changshu, et al.
Published: (2023)

Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain
by: Min, Marcus J., et al.
Published: (2023)

EditLord: Learning Code Transformation Rules for Code Editing
by: Li, Weichen, et al.
Published: (2025)

CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs
by: Manh, Dung Nguyen, et al.
Published: (2024)

Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
by: Kim, Myeongsoo, et al.
Published: (2026)

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level
by: Li, Jia, et al.
Published: (2026)

KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution
by: Mathai, Alex, et al.
Published: (2024)

REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
by: Pabba, Anvith, et al.
Published: (2025)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
by: Kuhar, Sachit, et al.
Published: (2024)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

An Execution-Verified Multi-Language Benchmark for Code Semantic Reasoning
by: Li, Yikun, et al.
Published: (2026)

COFFE: A Code Efficiency Benchmark for Code Generation
by: Peng, Yun, et al.
Published: (2025)

Evaluating Code Reasoning Abilities of Large Language Models Under Real-World Settings
by: Liu, Changshu, et al.
Published: (2025)

CRQBench: A Benchmark of Code Reasoning Questions
by: Dinella, Elizabeth, et al.
Published: (2024)

Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
by: Ceka, Ira, et al.
Published: (2025)

RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices
by: Li, Jia, et al.
Published: (2026)

ConceptCoder: Improve Code Reasoning via Concept Learning
by: Rahman, Md Mahbubur, et al.
Published: (2026)

CodeFort: Robust Training for Code Generation Models
by: Zhang, Yuhao, et al.
Published: (2024)

SWE-Refactor: A Repository-Level Benchmark for Real-World LLM-Based Code Refactoring
by: Xu, Yisen, et al.
Published: (2026)

Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection
by: Ceka, Ira, et al.
Published: (2024)

SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning
by: Peng, Jinjun, et al.
Published: (2026)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

SeRe: A Security-Related Code Review Dataset Aligned with Real-World Review Activities
by: Zhao, Zixiao, et al.
Published: (2026)

CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
by: Zhang, Kechi, et al.
Published: (2024)