Cover Image

StackEval: Benchmarking LLMs in Coding Assistance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shah, Nidhish, Genc, Zulkuf, Araci, Dogu
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CONCUR: Benchmarking LLMs for Concurrent Code Generation
by: Huang, Jue, et al.
Published: (2026)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
by: Petrukha, Ivan, et al.
Published: (2025)

Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM
by: Xia, Chunqiu Steven, et al.
Published: (2024)

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts
by: Zhang, Shudan, et al.
Published: (2024)

mcdok at SemEval-2026 Task 13: Finetuning LLMs for Detection of Machine-Generated Code
by: Skurla, Adam, et al.
Published: (2026)

LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs
by: Xia, Yunhui, et al.
Published: (2025)

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
by: Wu, Jiarong, et al.
Published: (2025)

CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs
by: Guo, Hanxi, et al.
Published: (2025)

A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation
by: Gu, Jian, et al.
Published: (2025)

Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study
by: Li, Yufei, et al.
Published: (2024)

NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness
by: Singhal, Manav, et al.
Published: (2024)

DependEval: Benchmarking LLMs for Repository Dependency Understanding
by: Du, Junjia, et al.
Published: (2025)

SmartEval: A Benchmark for Evaluating LLM-Generated Smart Contracts from Natural Language Specifications
by: Goel, Abhinav, et al.
Published: (2026)

Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs
by: Yang, Dayu, et al.
Published: (2025)

SelfCodeAlign: Self-Alignment for Code Generation
by: Wei, Yuxiang, et al.
Published: (2024)

IntentCoding: Amplifying User Intent in Code Generation
by: Fang, Zheng, et al.
Published: (2026)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

Rethinking Repetition Problems of LLMs in Code Generation
by: Dong, Yihong, et al.
Published: (2025)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

CODEMENV: Benchmarking Large Language Models on Code Migration
by: Cheng, Keyuan, et al.
Published: (2025)

CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences
by: Weyssow, Martin, et al.
Published: (2024)

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
by: Gong, Linyuan, et al.
Published: (2024)

How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data
by: Wang, Yejie, et al.
Published: (2024)

LLMs as Compiler for Arabic Programming Language
by: Sibaee, Serry, et al.
Published: (2024)

Learning Code Preference via Synthetic Evolution
by: Liu, Jiawei, et al.
Published: (2024)

Evaluating Language Models for Efficient Code Generation
by: Liu, Jiawei, et al.
Published: (2024)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

RepoQA: Evaluating Long Context Code Understanding
by: Liu, Jiawei, et al.
Published: (2024)

Towards Understanding What Code Language Models Learned
by: Ahmed, Toufique, et al.
Published: (2023)

Wisdom and Delusion of LLM Ensembles for Code Generation and Repair
by: Vallecillos-Ruiz, Fernando, et al.
Published: (2025)

GiFT: Gibbs Fine-Tuning for Code Generation
by: Li, Haochen, et al.
Published: (2025)

MetaLint: Easy-to-Hard Generalization for Code Linting
by: Naik, Atharva, et al.
Published: (2025)

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
by: Gong, Linyuan, et al.
Published: (2024)

Planning-Aware Code Infilling via Horizon-Length Prediction
by: Ding, Yifeng, et al.
Published: (2024)

Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
by: Riddell, Martin, et al.
Published: (2024)

Hybrid-Gym: Training Coding Agents to Generalize Across Tasks
by: Xie, Yiqing, et al.
Published: (2026)

Data Augmentation for Code Translation with Comparable Corpora and Multiple References
by: Xie, Yiqing, et al.
Published: (2023)

Position: Intelligent Coding Systems Should Write Programs with Justifications
by: Xu, Xiangzhe, et al.
Published: (2025)

Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning
by: Yang, Rem, et al.
Published: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
by: Chen, Yeheng, et al.
Published: (2026)