Cover Image

Evaluating Code Reasoning Abilities of Large Language Models Under Real-World Settings

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Changshu, Ghazanfari, Alireza, Chen, Yang, Jabbarvand, Reyhaneh
Format:	Preprint
Published:	2025
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models
by: Liu, Changshu, et al.
Published: (2025)

Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Models
by: Liu, Changshu, et al.
Published: (2025)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

Evaluating LLMs Code Reasoning Under Real-World Context
by: Liu, Changshu
Published: (2026)

Narrowing the Complexity Gap in the Evaluation of Large Language Models
by: Chen, Yang, et al.
Published: (2026)

A Generic Approach to Fix Test Flakiness in Real-World Projects
by: Chen, Yang, et al.
Published: (2024)

ReCodeAgent: A Multi-Agent Workflow for Language-agnostic Translation and Validation of Large-scale Repositories
by: Ibrahimzada, Ali Reza, et al.
Published: (2026)

Can Old Tests Do New Tricks for Resolving SWE Issues?
by: Chen, Yang, et al.
Published: (2025)

SPARC: Scenario Planning and Reasoning for Automated C Unit Test Generation
by: Chowdhury, Jaid Monwar, et al.
Published: (2026)

Challenging Bug Prediction and Repair Models with Synthetic Bugs
by: Ibrahimzada, Ali Reza, et al.
Published: (2023)

MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair
by: Ibrahimzada, Ali Reza, et al.
Published: (2025)

Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code
by: Pan, Rangeet, et al.
Published: (2023)

WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models
by: Yang, Chenyuan, et al.
Published: (2023)

Advancing Automated In-Isolation Validation in Repository-Level Code Translation
by: Ke, Kaiyao, et al.
Published: (2025)

Translating Large-Scale C Repositories to Idiomatic Rust
by: Dehghan, Saman, et al.
Published: (2025)

Evaluating Plan Compliance in Autonomous Programming Agents
by: Liu, Shuyang, et al.
Published: (2026)

EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits
by: Chi, Wayne, et al.
Published: (2025)

LeTI: Learning to Generate from Textual Interactions
by: Wang, Xingyao, et al.
Published: (2023)

CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
by: Tang, Lingxiao, et al.
Published: (2025)

AlphaTrans: A Neuro-Symbolic Compositional Approach for Repository-Level Code Translation and Validation
by: Ibrahimzada, Ali Reza, et al.
Published: (2024)

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
by: Xu, Jingxuan, et al.
Published: (2025)

INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair
by: Wang, Hanbin, et al.
Published: (2023)

Can Large Language Models Serve as Evaluators for Code Summarization?
by: Wu, Yang, et al.
Published: (2024)

R2C2-Coder: Enhancing and Benchmarking Real-world Repository-level Code Completion Abilities of Code Large Language Models
by: Deng, Ken, et al.
Published: (2024)

A Study on Thinking Patterns of Large Reasoning Models in Code Generation
by: Halim, Kevin, et al.
Published: (2025)

CFCEval: Evaluating Security Aspects in Code Generated by Large Language Models
by: Cheng, Cheng, et al.
Published: (2025)

Improving the Ability of Pre-trained Language Model by Imparting Large Language Model's Experience
by: Yin, Xin, et al.
Published: (2024)

SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories?
by: He, Xinyi, et al.
Published: (2025)

SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications
by: Ma, Lezhi, et al.
Published: (2024)

Automated Code Editing with Search-Generate-Modify
by: Liu, Changshu, et al.
Published: (2023)

CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases
by: Hoang, Anh Nguyen, et al.
Published: (2025)

Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation
by: Sun, Zhihong, et al.
Published: (2025)

Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning
by: Yang, Rem, et al.
Published: (2025)

The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers
by: Mozannar, Hussein, et al.
Published: (2024)

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
by: Hu, Ruida, et al.
Published: (2024)

Hotfixing Large Language Models for Code
by: Yang, Zhou, et al.
Published: (2024)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

Hierarchical Evaluation of Software Design Capabilities of Large Language Models of Code
by: Saad, Mootez, et al.
Published: (2025)

Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
by: Chen, Simin, et al.
Published: (2025)

Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems
by: Zhao, Chenyu, et al.
Published: (2025)