Cover Image

RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	LaBash, Beck, Rosedale, August, Reents, Alex, Negritto, Lucas, Wiel, Colin
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Improving Generative Inverse Design of Rectangular Patch Antennas with Test Time Optimization
by: LaBash, Beck, et al.
Published: (2025)

GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models
by: Jiang, Pengcheng, et al.
Published: (2024)

ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
by: Tang, Xiangru, et al.
Published: (2023)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
by: Guo, Jiawei, et al.
Published: (2024)

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases
by: Liu, Xiangyan, et al.
Published: (2024)

DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale
by: Zhang, Linghao, et al.
Published: (2025)

SWE-QA: Can Language Models Answer Repository-level Code Questions?
by: Peng, Weihan, et al.
Published: (2025)

CodeSimpleQA: Scaling Factuality in Code Large Language Models
by: Yang, Jian, et al.
Published: (2025)

InstructCoder: Instruction Tuning Large Language Models for Code Editing
by: Li, Kaixin, et al.
Published: (2023)

UniCoder: Scaling Code Large Language Model via Universal Code
by: Sun, Tao, et al.
Published: (2024)

On the Robustness of Editing Large Language Models
by: Ma, Xinbei, et al.
Published: (2024)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

Code Simulation Challenges for Large Language Models
by: La Malfa, Emanuele, et al.
Published: (2024)

Mitigating Gender Bias in Code Large Language Models via Model Editing
by: Qin, Zhanyue, et al.
Published: (2024)

R2C2-Coder: Enhancing and Benchmarking Real-world Repository-level Code Completion Abilities of Code Large Language Models
by: Deng, Ken, et al.
Published: (2024)

The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models
by: He, Muyu, et al.
Published: (2025)

Uncovering Overfitting in Large Language Model Editing
by: Zhang, Mengqi, et al.
Published: (2024)

Stable Knowledge Editing in Large Language Models
by: Wei, Zihao, et al.
Published: (2024)

Investigating Model Editing for Unlearning in Large Language Models
by: Hossain, Shariqah, et al.
Published: (2025)

Evaluating and Adapting Large Language Models to Represent Folktales in Low-Resource Languages
by: Meaney, JA, et al.
Published: (2024)

CodeS: Natural Language to Code Repository via Multi-Layer Sketch
by: Zan, Daoguang, et al.
Published: (2024)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

TokenSmith: Streamlining Data Editing, Search, and Inspection for Large-Scale Language Model Training and Interpretability
by: Khan, Mohammad Aflah, et al.
Published: (2025)

Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ
by: Holtermann, Carolin, et al.
Published: (2024)

APPLS: Evaluating Evaluation Metrics for Plain Language Summarization
by: Guo, Yue, et al.
Published: (2023)

TAXI: Evaluating Categorical Knowledge Editing for Language Models
by: Powell, Derek, et al.
Published: (2024)

Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics
by: Al-Khalili, Zena, et al.
Published: (2025)

Knowledge Graph Enhanced Large Language Model Editing
by: Zhang, Mengqi, et al.
Published: (2024)

Neuron-Level Sequential Editing for Large Language Models
by: Jiang, Houcheng, et al.
Published: (2024)

Benchmarking and Rethinking Knowledge Editing for Large Language Models
by: He, Guoxiu, et al.
Published: (2025)

Neighboring Perturbations of Knowledge Editing on Large Language Models
by: Ma, Jun-Yu, et al.
Published: (2024)

Probing then Editing Response Personality of Large Language Models
by: Ju, Tianjie, et al.
Published: (2025)

Disentangling Knowledge Representations for Large Language Model Editing
by: Zhang, Mengqi, et al.
Published: (2025)

Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models
by: Hsueh, Cheng-Hsun, et al.
Published: (2024)

RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing
by: Xie, Yiqing, et al.
Published: (2025)

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
by: Hada, Rishav, et al.
Published: (2023)

NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
by: Ding, Jingzhe, et al.
Published: (2025)

Improving Code Localization with Repository Memory
by: Wang, Boshi, et al.
Published: (2025)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula
by: Lucy, Li, et al.
Published: (2024)