Cover Image

Output Format Biases in the Evaluation of Large Language Models for Code Translation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Macedo, Marcos, Tian, Yuan, Cogo, Filipe R., Adams, Bram
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation
by: Macedo, Marcos, et al.
Published: (2024)

Understanding Prompt Management in GitHub Repositories: A Call for Best Practices
by: Li, Hao, et al.
Published: (2025)

Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
by: Li, Yuangang, et al.
Published: (2026)

Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation
by: Hamedi, Parisa, et al.
Published: (2025)

Exploring and Unleashing the Power of Large Language Models in Automated Code Translation
by: Yang, Zhen, et al.
Published: (2024)

Evaluating Large Language Models for Code Review
by: Cihan, Umut, et al.
Published: (2025)

Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing
by: Dhruv, Akash, et al.
Published: (2024)

A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
by: Wu, Yixi, et al.
Published: (2024)

Towards Semantic Versioning of Open Pre-trained Language Model Releases on Hugging Face
by: Ajibode, Adekunle, et al.
Published: (2024)

Large Language Models as Test Case Generators: Performance Evaluation and Enhancement
by: Li, Kefan, et al.
Published: (2024)

Task Abstention for Large Language Models in Code Generation
by: Zhou, Yanke, et al.
Published: (2026)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
by: He, Mengliang, et al.
Published: (2025)

Exploringand Unleashing the Power of Large Language Models in CI/CD Configuration Translation
by: Wang, Chong, et al.
Published: (2025)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
by: Xiong, Zhanhang, et al.
Published: (2025)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
by: Padwal, Vedant
Published: (2026)

Revisiting the Role of Natural Language Code Comments in Code Translation
by: Gupta, Monika, et al.
Published: (2026)

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
by: Meaden, James, et al.
Published: (2025)

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
by: Xu, Jingxuan, et al.
Published: (2025)

Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
by: Christakis, Nicholas, et al.
Published: (2025)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
by: Zhang, Tanghaoran, et al.
Published: (2026)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
by: Yin, Wenjing, et al.
Published: (2025)

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?
by: Chen, QiHong, et al.
Published: (2024)

HAFixAgent: History-Aware Program Repair Agent
by: Shi, Yu, et al.
Published: (2025)

Enhancing High-Quality Code Generation in Large Language Models with Comparative Prefix-Tuning
by: Jiang, Yuan, et al.
Published: (2025)

WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
by: Lei, Xinping, et al.
Published: (2026)

Skeleton-Guided-Translation: A Benchmarking Framework for Code Repository Translation with Fine-Grained Quality Evaluation
by: Zhang, Xing, et al.
Published: (2025)

RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation
by: Lin, Feng, et al.
Published: (2025)

Evaluating Large Language Models for Functional and Maintainable Code in Industrial Settings: A Case Study at ASML
by: Mundhra, Yash, et al.
Published: (2025)

Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation
by: Bhattarai, Manish, et al.
Published: (2024)

OmniLLP: Enhancing LLM-based Log Level Prediction with Context-Aware Retrieval
by: Ouatiti, Youssef Esseddiq, et al.
Published: (2025)

GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
by: Hou, Shuyang, et al.
Published: (2024)

Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements
by: Abtahi, Seyed Moein, et al.
Published: (2025)

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages
by: Kammakomati, Mehant, et al.
Published: (2024)

Personality-Guided Code Generation Using Large Language Models
by: Guo, Yaoqi, et al.
Published: (2024)

Bugs in Large Language Models Generated Code: An Empirical Study
by: Tambon, Florian, et al.
Published: (2024)

Assessing the Code Clone Detection Capability of Large Language Models
by: Zhang, Zixian, et al.
Published: (2024)

Large Language Model Guided Self-Debugging Code Generation
by: Adnan, Muntasir, et al.
Published: (2025)

Post-Incorporating Code Structural Knowledge into Pretrained Models via ICL for Code Translation
by: Du, Yali, et al.
Published: (2025)

Squeez: Task-Conditioned Tool-Output Pruning for Coding Agents
by: Kovács, Ádám
Published: (2026)

Towards Evaluation Engineering: An Empirical Study of ML Evaluation Harnesses in the Wild
by: Zhao, Zhimin, et al.
Published: (2026)