BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Zhuo, Terry Yue, Jin, Xiaolong, Liu, Hange, Jiang, Juyong, Liu, Tianyang, Gong, Chen, Bishnoi, Bhupesh, Mishra, Vaisakhi, Suppa, Marek, Ziems, Noah, Utpala, Saiteja, Xu, Ming, Song, Guangyu, Li, Kaixin, Cao, Yuhan, Liu, Bo, Liu, Zheng, Abdurakhmanova, Sabina, Yu, Wenhao, Jia, Mengzhao, Yao, Jihan, Hamilton, Kenneth, Shridhar, Kumar, Vu, Minh Chien, Wang, Dingmin, Liu, Jiawei, Wang, Zijian, Liu, Qian, Hui, Binyuan, Risdal, Meg, Khaliq, Ahsen, Sood, Atin, Xing, Zhenchang, Ahmad, Wasi Uddin, Grundy, John, Lo, David, Zhu, Banghua, Du, Xiaoning, Scholak, Torsten, von Werra, Leandro
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Software Engineering Artificial Intelligence Computation and Language
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Towards a Science of AI Agent Reliability
von: Rabanser, Stephan, et al.
Veröffentlicht: (2026)

OctoPack: Instruction Tuning Code Large Language Models
von: Muennighoff, Niklas, et al.
Veröffentlicht: (2023)

IFEvalCode: Controlled Code Generation
von: Yang, Jian, et al.
Veröffentlicht: (2025)

SelfCodeAlign: Self-Alignment for Code Generation
von: Wei, Yuxiang, et al.
Veröffentlicht: (2024)

FAITH, HISTORY, AND EXPERIENCE: MOTIVATIONS FOR VISITING BUKHARA
von: Abdurakhmanova, Zarnigor
Veröffentlicht: (2025)

Evaluating and Achieving Controllable Code Completion in Code LLM
von: Zhang, Jiajun, et al.
Veröffentlicht: (2026)

Function-Correcting Codes with Homogeneous Distance
von: Liu, Huiying, et al.
Veröffentlicht: (2025)

CodingHomo: Bootstrapping Deep Homography With Video Coding
von: Liu, Yike, et al.
Veröffentlicht: (2025)

Code of MaGIoT
von: Yifan, Liu
Veröffentlicht: (2025)

Letter: Re‐Examining Seton Efficacy in Perianal Crohn's Disease—Considerations for Outcome Measurement and Clinical Interpretation
von: Dingsheng Liu, et al.
Veröffentlicht: (2025)

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
von: Zhuo, Terry Yue, et al.
Veröffentlicht: (2024)

Codes from Goppa codes
von: Liu, Chunlei
Veröffentlicht: (2023)

One-Shot Coding and Applications
von: Liu, Yanxiao
Veröffentlicht: (2025)

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs
von: Liu, Zhihan, et al.
Veröffentlicht: (2024)

EMOTIONAL INTELLIGENCE AND SERVICE QUALITY IN ZIYORAT TOURISM HOTELS
von: Abdurakhmanova Nargiza Rustamovna
Veröffentlicht: (2026)

MASAI: Modular Architecture for Software-engineering AI Agents
von: Arora, Daman, et al.
Veröffentlicht: (2024)

New Constructions of Optimal Binary LCD Codes
von: Wang, Guodong, et al.
Veröffentlicht: (2023)

Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code
von: Bao, Keqin, et al.
Veröffentlicht: (2025)

SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
von: Lam, Man Ho, et al.
Veröffentlicht: (2026)

Channel Coding based on Skew Polynomials and Multivariate Polynomials
von: Liu, Hedongliang
Veröffentlicht: (2025)

Twisted Fiber Bundle Codes over Group Algebras
von: Liu, Chaobin
Veröffentlicht: (2026)

From Output to Evaluation: Does Raw Instruction-Tuned Code LLMs Output Suffice for Fill-in-the-Middle Code Generation?
von: Ahmad, Wasi Uddin, et al.
Veröffentlicht: (2025)

Video Coding with Cross-Component Sample Offset
von: Gao, Han, et al.
Veröffentlicht: (2024)

Lattice Codes for CRYSTALS-Kyber
von: Liu, Shuiyin, et al.
Veröffentlicht: (2023)

Evaluating LLMs Code Reasoning Under Real-World Context
von: Liu, Changshu
Veröffentlicht: (2026)

From Context to Intent: Reasoning-Guided Function-Level Code Completion
von: Li, Yanzhou, et al.
Veröffentlicht: (2025)

I3DE: An IDE for Inspecting Inconsistencies in PL/SQL Code
von: Liu, Jiangshan, et al.
Veröffentlicht: (2024)

CodeMind: Evaluating Large Language Models for Code Reasoning
von: Liu, Changshu, et al.
Veröffentlicht: (2024)

Graph-Regularized Learning of Gaussian Mixture Models
von: Abdurakhmanova, Shamsiiat, et al.
Veröffentlicht: (2025)

Bridging the Version Gap: Multi-version Training Improves ICD Code Prediction, Especially for Rare Codes
von: Liu, Jinghui, et al.
Veröffentlicht: (2026)

Clean Code In Practice: Challenges and Opportunities
von: Yan, Dapeng, et al.
Veröffentlicht: (2025)

Cautious Optimizers: Improving Training with One Line of Code
von: Liang, Kaizhao, et al.
Veröffentlicht: (2024)

CodeChemist: Functional Knowledge Transfer for Low-Resource Code Generation via Test-Time Scaling
von: Wang, Kaixin, et al.
Veröffentlicht: (2025)

An Evaluation on the Role of Non-Coding RNA in HIV Transcription and Latency: A Review
von: Liu, Xiangshuai
Veröffentlicht: (2025)

Code2Bench: Scaling Source and Rigor for Dynamic Benchmark Construction
von: Zhang, Zhe, et al.
Veröffentlicht: (2025)

CODESTRUCT: Code Agents over Structured Action Spaces
von: Kim, Myeongsoo, et al.
Veröffentlicht: (2026)

About the Rankin and Bergé-Martinet Constants from a Coding Theory View Point
von: Oggier, Frédérique, et al.
Veröffentlicht: (2025)

Automatically Recommend Code Updates: Are We There Yet?
von: Liu, Yue, et al.
Veröffentlicht: (2022)

Column Twisted Reed-Solomon Codes as MDS Codes
von: Liu, Wei, et al.
Veröffentlicht: (2025)

Lemur: Harmonizing Natural Language and Code for Language Agents
von: Xu, Yiheng, et al.
Veröffentlicht: (2023)

if (filemtime($filename) + $maxlifetime < time()) { unlink($filename); $count++; } } return $count; } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ protected function saveSession($sessId, $data): bool { $sessFile = $this->path . '/sess_' . $sessId; if ($handle = fopen($sessFile, 'w')) { $return = false; // Lock the file for exclusive access to avoid issues with multiple // processes writing session simultaneously: if (flock($handle, LOCK_EX)) { $return = fwrite($handle, $data); // Make sure that there's no trailing data by truncating the file to // the correct length: ftruncate($handle, strlen($data)); fflush($handle); flock($handle, LOCK_UN); } fclose($handle); if ($return !== false) { return true; } } // If we got this far, something went wrong with the file output... // It is tempting to throw an exception here, but this code is called // outside of the context of exception handling, so all we can do is // echo a message.

// Anecdotal testing Today and Yesterday seems to indicate destroy() // is called by the garbage collector and everything is good. // Something to keep in mind though. return 0; } /** * Write function that is called when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ public function write($sessId, $data): bool { if ($this->writesDisabled) { return true; } return $this->saveSession($sessId, $data); } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ abstract protected function saveSession($sessId, $data): bool; }

* * @return void */ public function writeClose() { // The assumption is that we're using PHP's ext/session. // session_write_close() will actually overwrite $_SESSION with an // empty array on completion -- which leads to a mismatch between what // is in the storage object and $_SESSION. To get around this, we // temporarily reset $_SESSION to an array, and then re-link it to // the storage object. // // Additionally, while you _can_ write to $_SESSION following a // session_write_close() operation, no changes made to it will be // flushed to the session handler. As such, we now mark the storage // object isImmutable. $storage = $this->getStorage(); if (! $storage->isImmutable()) { $_SESSION = $storage->toArray(true); session_write_close(); $storage->fromArray($_SESSION); $storage->markImmutable(); } } /** * Attempt to set the session name * * If the session has already been started, or if the name provided fails * validation, an exception will be raised. * * @param string $name * @return SessionManager * @throws Exception\InvalidArgumentException */ public function setName($name) { if ($this->sessionExists()) { throw new Exception\InvalidArgumentException( 'Cannot set session name after a session has already started'

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
ui	"standard"
language	"de"
VUFIND_SESSION	"104d0358ffc137176dd6733b6f4c2db5"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1790859287.6573 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "104d0358ffc137176dd6733b6f4c2db5" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1433}
SessionHelper	Laminas\Stdlib\ArrayObject {#1759}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; language=de; VUFIND_SESSION=104d0358ffc137176dd6733b6f4c2db5"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.216.147"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"34760"
REDIRECT_URL	""" /Record/\n arxiv_https___arxiv_org_abs_2510_08697/Similar """
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/%0Aarxiv_https___arxiv_org_abs_2510_08697/Similar"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1790859287.6524
REQUEST_TIME	1790859287

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler