Copertina

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wu, Jiarong, Chen, Songqiang, Cao, Jialun, Lo, Hau Ching, Cheung, Shing-Chi
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Software Engineering Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can Emulating Semantic Translation Help LLMs with Code Translation? A Study Based on Pseudocode
di: Chen, Songqiang, et al.
Pubblicazione: (2025)

CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit
di: Cao, Jialun, et al.
Pubblicazione: (2024)

What Builds Effective In-Context Examples for Code Generation?
di: Li, Dongze, et al.
Pubblicazione: (2025)

When LLMs Meet API Documentation: Can Retrieval Augmentation Aid Code Generation Just as It Helps Developers?
di: Chen, Jingyi, et al.
Pubblicazione: (2025)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
di: Cao, Jialun, et al.
Pubblicazione: (2024)

MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing
di: Xu, Congying, et al.
Pubblicazione: (2024)

Can Large Language Models Model Programs Formally?
di: Chen, Zhiyong, et al.
Pubblicazione: (2026)

Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
di: Cao, Jialun, et al.
Pubblicazione: (2024)

RulER: Automated Rule-Based Semantic Error Localization and Repair for Code Translation
di: Jin, Shuo, et al.
Pubblicazione: (2025)

ModelWisdom: An Integrated Toolkit for TLA+ Model Visualization, Digest and Repair
di: Chen, Zhiyong, et al.
Pubblicazione: (2026)

MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis
di: Xu, Congying, et al.
Pubblicazione: (2026)

Word Closure-Based Metamorphic Testing for Machine Translation
di: Xie, Xiaoyuan, et al.
Pubblicazione: (2023)

LSPFuzz: Hunting Bugs in Language Servers
di: Zhu, Hengcheng, et al.
Pubblicazione: (2025)

Multi-Agent Systems for Dataset Adaptation in Software Engineering: Capabilities, Limitations, and Future Directions
di: Chen, Jingyi, et al.
Pubblicazione: (2025)

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
di: Zhu, Qiming, et al.
Pubblicazione: (2024)

StackEval: Benchmarking LLMs in Coding Assistance
di: Shah, Nidhish, et al.
Pubblicazione: (2024)

ReuseDroid: A VLM-empowered Android UI Test Migrator Boosted by Active Feedback
di: Li, Xiaolei, et al.
Pubblicazione: (2025)

Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
di: Zhu, Qiming, et al.
Pubblicazione: (2025)

Automatic Build Repair for Test Cases using Incompatible Java Versions
di: Mak, Ching Hang, et al.
Pubblicazione: (2024)

Rigor, Reliability, and Reproducibility Matter: A Decade-Scale Survey of 572 Code Benchmarks
di: Cao, Jialun, et al.
Pubblicazione: (2025)

DependEval: Benchmarking LLMs for Repository Dependency Understanding
di: Du, Junjia, et al.
Pubblicazione: (2025)

FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
di: Dai, Dekun, et al.
Pubblicazione: (2025)

CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems
di: Lyu, Zongyi, et al.
Pubblicazione: (2026)

EmbedAgent: Benchmarking Large Language Models in Embedded System Development
di: Xu, Ruiyang, et al.
Pubblicazione: (2025)

MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases
di: Xu, Congying, et al.
Pubblicazione: (2023)

Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation
di: Lyu, Zongyi, et al.
Pubblicazione: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
di: Chen, Yeheng, et al.
Pubblicazione: (2026)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
di: Yang, Jialin, et al.
Pubblicazione: (2025)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
di: Wang, Jiexin, et al.
Pubblicazione: (2024)

HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization
di: Peng, Qiwei, et al.
Pubblicazione: (2024)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
di: Li, Jia, et al.
Pubblicazione: (2024)

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
di: Zhao, Yuwei, et al.
Pubblicazione: (2024)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
di: Liu, Kaiyuan, et al.
Pubblicazione: (2025)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
di: Petrukha, Ivan, et al.
Pubblicazione: (2025)

Understanding and Characterizing Mock Assertions in Unit Tests
di: Zhu, Hengcheng, et al.
Pubblicazione: (2025)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
di: Chen, Zaoyu, et al.
Pubblicazione: (2026)

Is Your Benchmark (Still) Useful? Dynamic Benchmarking for Code Language Models
di: Guan, Batu, et al.
Pubblicazione: (2025)

LLMs in Mobile Apps: Practices, Challenges, and Opportunities
di: Hau, Kimberly, et al.
Pubblicazione: (2025)

NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition
di: Deng, Le, et al.
Pubblicazione: (2025)

IndicEval-XL: Bridging Linguistic Diversity in Code Generation Across Indic Languages
di: Singh, Ujjwal, et al.
Pubblicazione: (2025)