Cover Image

SIMCOPILOT: Evaluating Large Language Models for Copilot-Style Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jiang, Mingchao, Jain, Abhinav, Zorek, Sophia, Jermaine, Chris
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Programming Languages Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Large Language Models for Multilingual Code Intelligence: A Survey
by: Jiang, Chao, et al.
Published: (2026)

Language Models for Code Completion: A Practical Evaluation
by: Izadi, Maliheh, et al.
Published: (2024)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
by: Cao, Jialun, et al.
Published: (2024)

NExT: Teaching Large Language Models to Reason about Code Execution
by: Ni, Ansong, et al.
Published: (2024)

Finding Missed Code Size Optimizations in Compilers using LLMs
by: Italiano, Davide, et al.
Published: (2024)

Large Language Models for Code Summarization
by: Szalontai, Balázs, et al.
Published: (2024)

Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
by: Cassano, Federico, et al.
Published: (2023)

A Multi-Expert Large Language Model Architecture for Verilog Code Generation
by: Nadimi, Bardia, et al.
Published: (2024)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

Evaluating Quantized Large Language Models for Code Generation on Low-Resource Language Benchmarks
by: Nyamsuren, Enkhbold
Published: (2024)

Large Language Models for Code: Security Hardening and Adversarial Testing
by: He, Jingxuan, et al.
Published: (2023)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

Assured Automatic Programming via Large Language Models
by: Mirchev, Martin, et al.
Published: (2024)

Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning
by: Yang, Rem, et al.
Published: (2025)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
by: Petrukha, Ivan, et al.
Published: (2025)

Correctness-Guaranteed Code Generation via Constrained Decoding
by: Li, Lingxiao, et al.
Published: (2025)

DeepCode AI Fix: Fixing Security Vulnerabilities with Large Language Models
by: Berabi, Berkay, et al.
Published: (2024)

WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models
by: Yang, Chenyuan, et al.
Published: (2023)

Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars
by: Melcer, Daniel, et al.
Published: (2024)

Grounding Data Science Code Generation with Input-Output Specifications
by: Wen, Yeming, et al.
Published: (2024)

MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks
by: Li, Jingyao, et al.
Published: (2023)

Automatically Testing Functional Properties of Code Translation Models
by: Eniser, Hasan Ferit, et al.
Published: (2023)

MLCPD: A Unified Multi-Language Code Parsing Dataset with Universal AST Schema
by: Gajjar, Jugal, et al.
Published: (2025)

Evaluating Language Models for Efficient Code Generation
by: Liu, Jiawei, et al.
Published: (2024)

Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates
by: Hooda, Ashish, et al.
Published: (2024)

MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
by: Kadosh, Tal, et al.
Published: (2023)

Zero-Shot RTL Code Generation with Attention Sink Augmented Large Language Models
by: Sandal, Selim, et al.
Published: (2024)

Neural Models for Source Code Synthesis and Completion
by: Niyogi, Mitodru
Published: (2024)

Strengthening Programming Comprehension in Large Language Models through Code Generation
by: Ren, Xiaoning, et al.
Published: (2025)

Bootstrapping Fuzzers for Compilers of Low-Resource Language Dialects Using Language Models
by: Vaidya, Sairam, et al.
Published: (2025)

Guess & Sketch: Language Model Guided Transpilation
by: Lee, Celine, et al.
Published: (2023)

Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
by: Riddell, Martin, et al.
Published: (2024)

Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing
by: Wei, Jiayi, et al.
Published: (2023)

PerfRL: A Small Language Model Framework for Efficient Code Optimization
by: Duan, Shukai, et al.
Published: (2023)

GraphMend: Code Transformations for Fixing Graph Breaks in PyTorch 2
by: Kashmira, Savini, et al.
Published: (2025)

ChatDBG: Augmenting Debugging with Large Language Models
by: Levin, Kyla H., et al.
Published: (2024)

Large Language Models Synergize with Automated Machine Learning
by: Xu, Jinglue, et al.
Published: (2024)

CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
by: Wang, Xinchen, et al.
Published: (2025)

From Effectiveness to Efficiency: Uncovering Linguistic Bias in Large Language Model-based Code Generation
by: Jiang, Weipeng, et al.
Published: (2024)

SmartEval: A Benchmark for Evaluating LLM-Generated Smart Contracts from Natural Language Specifications
by: Goel, Abhinav, et al.
Published: (2026)