Copertina

AXIOM: Benchmarking LLM-as-a-Judge for Code via Rule-Based Perturbation and Multisource Quality Calibration

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Ruiqi, Wang, Xinchen, Gao, Cuiyun, Chong, Chun Yong, Xia, Xin, Liao, Qing
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Software Engineering Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering
di: Wang, Ruiqi, et al.
Pubblicazione: (2025)

ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code
di: Feng, Jia, et al.
Pubblicazione: (2024)

An Empirical Study of Knowledge Distillation for Code Understanding Tasks
di: Wang, Ruiqi, et al.
Pubblicazione: (2025)

Game Rewards Vulnerabilities: Software Vulnerability Detection with Zero-Sum Game and Prototype Learning
di: Wen, Xin-Cheng, et al.
Pubblicazione: (2024)

ReposVul: A Repository-Level High-Quality Vulnerability Dataset
di: Wang, Xinchen, et al.
Pubblicazione: (2024)

Integrating Rules and Semantics for LLM-Based C-to-Rust Translation
di: Luo, Feng, et al.
Pubblicazione: (2025)

A Systematic Literature Review of Code Hallucinations in LLMs: Characterization, Mitigation Methods, Challenges, and Future Directions for Reliable AI
di: Gao, Cuiyun, et al.
Pubblicazione: (2025)

A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How
di: Wang, Chaozheng, et al.
Pubblicazione: (2024)

Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
di: Hu, Ruida, et al.
Pubblicazione: (2025)

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
di: Hu, Ruida, et al.
Pubblicazione: (2026)

MLLM-Based UI2Code Automation Guided by UI Layout Information
di: Wu, Fan, et al.
Pubblicazione: (2025)

Bridge and Hint: Extending Pre-trained Language Models for Long-Range Code
di: Chen, Yujia, et al.
Pubblicazione: (2024)

CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
di: Hu, Ruida, et al.
Pubblicazione: (2024)

CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
di: Wang, Xinchen, et al.
Pubblicazione: (2025)

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
di: Wu, Fan, et al.
Pubblicazione: (2026)

APIGen: Generative API Method Recommendation
di: Chen, Yujia, et al.
Pubblicazione: (2024)

Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
di: Wang, Xinchen, et al.
Pubblicazione: (2026)

The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
di: Gao, Shuzheng, et al.
Pubblicazione: (2025)

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
di: Hu, Ruida, et al.
Pubblicazione: (2024)

Repo2Run: Automated Building Executable Environment for Code Repository at Scale
di: Hu, Ruida, et al.
Pubblicazione: (2025)

SEER: Enhancing Chain-of-Thought Code Generation through Self-Exploring Deep Reasoning
di: Gao, Shuzheng, et al.
Pubblicazione: (2025)

A Roadmap on Modern Code Review: Challenges and Opportunities
di: Yang, Zezhou, et al.
Pubblicazione: (2024)

VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection
di: Wen, Xin-Cheng, et al.
Pubblicazione: (2024)

An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities
di: Yang, Zezhou, et al.
Pubblicazione: (2025)

What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?
di: Gao, Shuzheng, et al.
Pubblicazione: (2023)

Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMs
di: Chen, Yujia, et al.
Pubblicazione: (2026)

Learning in the Wild: Towards Leveraging Unlabeled Data for Effectively Tuning Pre-trained Code Models
di: Gao, Shuzheng, et al.
Pubblicazione: (2024)

The Current Challenges of Software Engineering in the Era of Large Language Models
di: Gao, Cuiyun, et al.
Pubblicazione: (2024)

Repository-Level Graph Representation Learning for Enhanced Security Patch Detection
di: Wen, Xin-Cheng, et al.
Pubblicazione: (2024)

When Model Editing Meets Service Evolution: A Knowledge-Update Perspective for Service Recommendation
di: Fan, Guodong, et al.
Pubblicazione: (2026)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
di: Jiang, Hongchao, et al.
Pubblicazione: (2025)

SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
di: Zhan, Zexun, et al.
Pubblicazione: (2025)

EvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
di: Wen, Xin-Cheng, et al.
Pubblicazione: (2024)

On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
di: Feng, Jia, et al.
Pubblicazione: (2026)

Uncovering Weaknesses in Neural Code Generation
di: Lian, Xiaoli, et al.
Pubblicazione: (2024)

Deep Learning Based Code Generation Methods: Literature Review
di: Yang, Zezhou, et al.
Pubblicazione: (2023)

Search-Based LLMs for Code Optimization
di: Gao, Shuzheng, et al.
Pubblicazione: (2024)

AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
di: Wang, Xinchen, et al.
Pubblicazione: (2024)

Code Digital Twin: Empowering LLMs with Tacit Knowledge for Complex Software Development
di: Peng, Xin, et al.
Pubblicazione: (2025)

A Closer Look into Transformer-Based Code Intelligence Through Code Transformation: Challenges and Opportunities
di: Li, Yaoxian, et al.
Pubblicazione: (2022)