Cover Image

CRQBench: A Benchmark of Code Reasoning Questions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dinella, Elizabeth, Chandra, Satish, Maniatis, Petros
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RubberDuckBench: A Benchmark for AI Coding Assistants
by: Mohammed, Ferida, et al.
Published: (2026)

On the Effectiveness of Modular Testing in EvoSuite
by: Dinella, Elizabeth
Published: (2026)

Agentic Code Reasoning
by: Ugare, Shubham, et al.
Published: (2026)

Program Structure Aware Precondition Generation
by: Dinella, Elizabeth, et al.
Published: (2023)

KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution
by: Mathai, Alex, et al.
Published: (2024)

Towards Verified Code Reasoning by LLMs
by: Sistla, Meghana, et al.
Published: (2025)

Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All
by: Huang, Chenxi, et al.
Published: (2026)

REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
by: Jha, Smriti, et al.
Published: (2026)

AI-Assisted Assessment of Coding Practices in Modern Code Review
by: Vijayvergiya, Manushree, et al.
Published: (2024)

CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs
by: Manh, Dung Nguyen, et al.
Published: (2024)

CoRE: A Fine-Grained Code Reasoning Benchmark Beyond Output Prediction
by: Gao, Jun, et al.
Published: (2026)

Defects4Log: Benchmarking LLMs for Logging Code Defect Detection and Reasoning
by: Wang, Xin, et al.
Published: (2025)

A Benchmark for Evaluating Repository-Level Code Agents with Intermediate Reasoning on Feature Addition Task
by: Liu, Shuhan, et al.
Published: (2026)

Beyond Code Snippets: Benchmarking LLMs on Repository-Level Question Answering
by: Alebachew, Yoseph Berhanu, et al.
Published: (2026)

COFFE: A Code Efficiency Benchmark for Code Generation
by: Peng, Yun, et al.
Published: (2025)

CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
by: Tang, Lingxiao, et al.
Published: (2025)

CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
by: Hu, Ruida, et al.
Published: (2024)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

UniCode: Augmenting Evaluation for Code Reasoning
by: Zheng, Xinyue, et al.
Published: (2025)

ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code
by: Feng, Jia, et al.
Published: (2024)

An Execution-Verified Multi-Language Benchmark for Code Semantic Reasoning
by: Li, Yikun, et al.
Published: (2026)

Robustness and Reasoning Fidelity of Large Language Models in Long-Context Code Question Answering
by: Maharaj, Kishan, et al.
Published: (2026)

GENCNIPPET: Automated Generation of Code Snippets for Supporting Programming Questions
by: Mondal, Saikat, et al.
Published: (2025)

Building an Internal Coding Agent at Zup: Lessons and Open Questions
by: Pinto, Gustavo, et al.
Published: (2026)

How is Google using AI for internal code migrations?
by: Nikolov, Stoyan, et al.
Published: (2025)

EffiReasonTrans: RL-Optimized Reasoning for Code Translation
by: Wang, Yanlin, et al.
Published: (2025)

Wink: Recovering from Misbehaviors in Coding Agents
by: Nanda, Rahul, et al.
Published: (2026)

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
by: Gu, Alex, et al.
Published: (2024)

Articulate but Wrong: Self-Review Failures in LLM-Based Code Modernization
by: Reddy, Gokul Chandra Purnachandra, et al.
Published: (2026)

Evaluating LLM-Generated Code: A Benchmark and Developer Study
by: Szych, Joanna, et al.
Published: (2026)

Development and Benchmarking of Multilingual Code Clone Detector
by: Zhu, Wenqing, et al.
Published: (2024)

SWE-Bench+: Enhanced Coding Benchmark for LLMs
by: Aleithan, Reem, et al.
Published: (2024)

Benchmarking and Studying the LLM-based Code Review
by: Zeng, Zhengran, et al.
Published: (2025)

ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers
by: Lin, Hannah, et al.
Published: (2025)

Can We Identify Stack Overflow Questions Requiring Code Snippets? Investigating the Cause & Effect of Missing Code Snippets
by: Mondal, Saikat, et al.
Published: (2024)

Benchmarking and Revisiting Code Generation Assessment: A Mutation-Based Approach
by: Wang, Longtian, et al.
Published: (2025)

DSCodeBench: A Realistic Benchmark for Data Science Code Generation
by: Ouyang, Shuyin, et al.
Published: (2025)

CoReQA: Uncovering Potentials of Language Models in Code Repository Question Answering
by: Chen, Jialiang, et al.
Published: (2025)

CLARC: C/C++ Benchmark for Robust Code Search
by: Wang, Kaicheng, et al.
Published: (2026)

Reading Between the Lines: Scalable User Feedback via Implicit Sentiment in Developer Prompts
by: Nam, Daye, et al.
Published: (2025)