Cover Image

Measuring memorization in RLHF for code completion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pappu, Aneesh, Porter, Billy, Shumailov, Ilia, Hayes, Jamie
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Overcoming linguistic barriers in code assistants: creating a QLoRA adapter to improve support for Russian-language code writing instructions
by: Pronin, C. B., et al.
Published: (2024)

Step Rejection Fine-Tuning: A Practical Distillation Recipe
by: Slinko, Igor, et al.
Published: (2026)

SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations
by: Wang, Shuaiqi, et al.
Published: (2026)

code_transformed: The Influence of Large Language Models on Code
by: Xu, Yuliang, et al.
Published: (2025)

Stealing User Prompts from Mixture of Experts
by: Yona, Itay, et al.
Published: (2024)

DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
by: Guo, Daya, et al.
Published: (2024)

Automatic Pull Request Description Generation Using LLMs: A T5 Model Approach
by: Sakib, Md Nazmus, et al.
Published: (2024)

Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation
by: Haider, Md. Asif, et al.
Published: (2024)

Test Code Generation for Telecom Software Systems using Two-Stage Generative Model
by: Nabeel, Mohamad, et al.
Published: (2024)

Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering
by: Ridnik, Tal, et al.
Published: (2024)

Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation
by: Mayer, Luis, et al.
Published: (2024)

Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks
by: Fazelnia, Mohamad, et al.
Published: (2024)

New Solutions on LLM Acceleration, Optimization, and Application
by: Huang, Yingbing, et al.
Published: (2024)

Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs
by: Iskander, Shadi, et al.
Published: (2024)

TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?
by: Ahmed, Toufique, et al.
Published: (2024)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

Learning Code Preference via Synthetic Evolution
by: Liu, Jiawei, et al.
Published: (2024)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
by: Gong, Linyuan, et al.
Published: (2024)

Aligning the Objective of LLM-based Program Repair
by: Xu, Junjielong, et al.
Published: (2024)

Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation
by: Ruiz, Fernando Vallecillos, et al.
Published: (2024)

Planning-Aware Code Infilling via Horizon-Length Prediction
by: Ding, Yifeng, et al.
Published: (2024)

SceneGenAgent: Precise Industrial Scene Generation with Coding Agent
by: Xia, Xiao, et al.
Published: (2024)

FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-scale Cloud Systems
by: Huang, Junjie, et al.
Published: (2024)

Exploring LLM-based Agents for Root Cause Analysis
by: Roy, Devjeet, et al.
Published: (2024)

Knowledge-aware Alert Aggregation in Large-scale Cloud Systems: a Hybrid Approach
by: Kuang, Jinxi, et al.
Published: (2024)

The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines
by: Martinez, Matias
Published: (2024)

Evaluating Language Models for Efficient Code Generation
by: Liu, Jiawei, et al.
Published: (2024)

CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences
by: Weyssow, Martin, et al.
Published: (2024)

Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study
by: Li, Yufei, et al.
Published: (2024)

LLMs as Compiler for Arabic Programming Language
by: Sibaee, Serry, et al.
Published: (2024)

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts
by: Zhang, Shudan, et al.
Published: (2024)

Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
by: Riddell, Martin, et al.
Published: (2024)

RepoQA: Evaluating Long Context Code Understanding
by: Liu, Jiawei, et al.
Published: (2024)

OptLLM: Optimal Assignment of Queries to Large Language Models
by: Liu, Yueyue, et al.
Published: (2024)

Evaluation and Improvement of Fault Detection for Large Language Models
by: Hu, Qiang, et al.
Published: (2024)

CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing
by: He, Xinyi, et al.
Published: (2024)

Defect Prediction with Content-based Features
by: Pham, Hung Viet, et al.
Published: (2024)

SelfCodeAlign: Self-Alignment for Code Generation
by: Wei, Yuxiang, et al.
Published: (2024)

StackEval: Benchmarking LLMs in Coding Assistance
by: Shah, Nidhish, et al.
Published: (2024)