Cover Image

Sphinx: Benchmarking and Modeling for LLM-Driven Pull Request Review

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Daoan, Zhang, Shuo, Jin, Zijian, Luo, Jiebo, Fu, Shengyu, Nallipogu, Elsie
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
by: Dai, Hankun, et al.
Published: (2025)

DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale
by: Zhang, Linghao, et al.
Published: (2025)

Learning to Commit: Generating Organic Pull Requests via Online Repository Memory
by: Li, Mo, et al.
Published: (2026)

Automatic Pull Request Description Generation Using LLMs: A T5 Model Approach
by: Sakib, Md Nazmus, et al.
Published: (2024)

Pull Requests as a Training Signal for Repo-Level Code Editing
by: Zhu, Qinglin, et al.
Published: (2026)

SWE-bench Goes Live!
by: Zhang, Linghao, et al.
Published: (2025)

Automatically Benchmarking LLM Code Agents through Agent-Driven Annotation and Evaluation
by: Fu, Lingyue, et al.
Published: (2025)

Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs
by: Yang, Dayu, et al.
Published: (2025)

DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Models
by: Kumarappan, Adarsh, et al.
Published: (2026)

Let's Make Every Pull Request Meaningful: An Empirical Analysis of Developer and Agentic Pull Requests
by: Yoshioka, Haruhiko, et al.
Published: (2026)

A Study of Library Usage in Agent-Authored Pull Requests
by: Twist, Lukas, et al.
Published: (2025)

FireBench: Evaluating Instruction Following in Enterprise and API-Driven LLM Applications
by: Zhang, Yunfan, et al.
Published: (2026)

The Value of Effective Pull Request Description
by: Pirouzkhah, Shirin, et al.
Published: (2026)

How Do Developers Use Code Suggestions in Pull Request Reviews?
by: Bouraffa, Abir, et al.
Published: (2025)

SWE-PRBench: Benchmarking AI Code Review Quality Against Pull Request Feedback
by: Kumar, Deepak
Published: (2026)

A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
by: Guo, Jiale, et al.
Published: (2025)

These Aren't the Reviews You're Looking For How Humans Review AI-Generated Pull Requests
by: Duma, Kacper, et al.
Published: (2026)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

Solver-Independent Automated Problem Formulation via LLMs for High-Cost Simulation-Driven Design
by: Li, Yuchen, et al.
Published: (2025)

Bugdar: AI-Augmented Secure Code Review for GitHub Pull Requests
by: Naulty, John, et al.
Published: (2025)

On the Footprints of Reviewer Bots Feedback on Agentic Pull Requests in OSS GitHub Repositories
by: Fatima, Syeda Kaneez, et al.
Published: (2026)

Quality Gatekeepers: Investigating the Effects ofCode Review Bots on Pull Request Activities
by: Wessel, Mairieli, et al.
Published: (2021)

Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
by: Yang, Kang, et al.
Published: (2025)

Insights into Security-Related AI-Generated Pull Requests
by: Rabbi, Md Fazle, et al.
Published: (2026)

The Impact of Documentation on Test Engagement in Pull Requests in OSS
by: Amore, Teal, et al.
Published: (2026)

Empirical Analysis of Pull Requests for Google Summer of Code
by: Popoola, Saheed
Published: (2024)

GitHub Actions: The Impact on the Pull Request Process
by: Wessel, Mairieli, et al.
Published: (2022)

ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agents
by: Li, Kenan, et al.
Published: (2026)

FASTRIC: Prompt Specification Language for Verifiable LLM Interactions
by: Jin, Wen-Long
Published: (2025)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
by: Chou, Jason, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

Dynamic Scaling of Unit Tests for Code Reward Modeling
by: Ma, Zeyao, et al.
Published: (2025)

Is Your Benchmark (Still) Useful? Dynamic Benchmarking for Code Language Models
by: Guan, Batu, et al.
Published: (2025)

From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests
by: Chowdhury, Kowshik, et al.
Published: (2026)

Benchmarking LLMs for Unit Test Generation from Real-World Functions
by: Huang, Dong, et al.
Published: (2025)

TestExplora: Benchmarking LLMs for Proactive Bug Discovery via Repository-Level Test Generation
by: Liu, Steven, et al.
Published: (2026)

ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents
by: Jeon, YoungHoon, et al.
Published: (2026)

Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study
by: Peralta, Sien Reeve O., et al.
Published: (2026)

The Quiet Contributions: Insights into AI-Generated Silent Pull Requests
by: Hasan, S M Mahedy, et al.
Published: (2026)