Image de couverture de livre

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cheng, Junhang, Liu, Fang, Li, Jia, Wu, Chengru, Jiang, Nanxiang, Zhang, Li
Format:	Preprint
Publié:	2026
Sujets:	Software Engineering Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

AdaptiveLLM: A Framework for Selecting Optimal Cost-Efficient LLM for Code-Generation Based on CoT Length
par: Cheng, Junhang, et autres
Publié: (2025)

Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code
par: Jiang, Nan, et autres
Publié: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
par: Li, Jia, et autres
Publié: (2024)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
par: Wang, Peiding, et autres
Publié: (2025)

FeatBench: Towards More Realistic Evaluation of Feature-level Code Generation
par: Chen, Haorui, et autres
Publié: (2025)

UA-Code-Bench: A Competitive Programming Benchmark for Evaluating LLM Code Generation in Ukrainian
par: Syromiatnikov, Mykyta, et autres
Publié: (2025)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
par: Yang, Jialin, et autres
Publié: (2025)

Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation
par: Fang, Sen, et autres
Publié: (2025)

LLMs Lean on Priors, Not Programming Language Semantics
par: Thimmaiah, Aditya, et autres
Publié: (2025)

AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?
par: Press, Ori, et autres
Publié: (2025)

Enhancing Automated Loop Invariant Generation for Complex Programs with Large Language Models
par: Liu, Ruibang, et autres
Publié: (2024)

EquiBench: Benchmarking Large Language Models' Reasoning about Program Semantics via Equivalence Checking
par: Wei, Anjiang, et autres
Publié: (2025)

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
par: Zhuo, Terry Yue, et autres
Publié: (2024)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
par: Chen, Zaoyu, et autres
Publié: (2026)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
par: Li, Jia, et autres
Publié: (2024)

To See is Not to Master: Teaching LLMs to Use Private Libraries for Code Generation
par: Zhang, Yitong, et autres
Publié: (2026)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
par: Duston, Titouan, et autres
Publié: (2025)

Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages
par: Zhang, William, et autres
Publié: (2024)

OSVBench: Benchmarking LLMs on Specification Generation Tasks for Operating System Verification
par: Li, Shangyu, et autres
Publié: (2025)

LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?
par: Zheng, Zihan, et autres
Publié: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
par: Jiang, Hongchao, et autres
Publié: (2025)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
par: Cao, Jialun, et autres
Publié: (2024)

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
par: Yan, Weixiang, et autres
Publié: (2023)

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
par: Yang, Jie, et autres
Publié: (2026)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
par: Zheng, Dewu, et autres
Publié: (2024)

LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks
par: Zhou, Xin, et autres
Publié: (2025)

AutoCode: LLMs as Problem Setters for Competitive Programming
par: Zhou, Shang, et autres
Publié: (2025)

PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models
par: Chen, Simin, et autres
Publié: (2024)

Dual-Language General-Purpose Self-Hosted Visual Language and new Textual Programming Language for Applications
par: Fayed, Mahmoud Samir
Publié: (2025)

Can LLMs Generate Reliable Test Case Generators? A Study on Competition-Level Programming Problems
par: Cao, Yuhan, et autres
Publié: (2025)

Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
par: Zhao, Qianhui, et autres
Publié: (2025)

DebugBench: Evaluating Debugging Capability of Large Language Models
par: Tian, Runchu, et autres
Publié: (2024)

Evaluating Quantized Large Language Models for Code Generation on Low-Resource Language Benchmarks
par: Nyamsuren, Enkhbold
Publié: (2024)

AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation
par: Sun, Zhensu, et autres
Publié: (2024)

BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks
par: Tu, Xinming, et autres
Publié: (2026)

BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
par: Diddee, Harshita, et autres
Publié: (2026)

LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language
par: Wang, Ming, et autres
Publié: (2024)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
par: Chou, Jason, et autres
Publié: (2025)

Multi-Programming Language Sandbox for LLMs
par: Dou, Shihan, et autres
Publié: (2024)

SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
par: Zhao, Bingchen, et autres
Publié: (2026)