Buchumschlag

EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Sharma, Aman, Chopra, Paras
Format:	Preprint
Veröffentlicht:	2026
Schlagworte:	Artificial Intelligence Machine Learning Software Engineering
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Evaluating Robustness of Large Language Models in Enterprise Applications: Benchmarks for Perturbation Consistency Across Formats and Languages
von: Bogavelli, Tara, et al.
Veröffentlicht: (2026)

Automatic Programming: Large Language Models and Beyond
von: Lyu, Michael R., et al.
Veröffentlicht: (2024)

EquiBench: Benchmarking Large Language Models' Reasoning about Program Semantics via Equivalence Checking
von: Wei, Anjiang, et al.
Veröffentlicht: (2025)

Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
von: Li, Yuangang, et al.
Veröffentlicht: (2026)

LangProp: A code optimization framework using Large Language Models applied to driving
von: Ishida, Shu, et al.
Veröffentlicht: (2024)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
von: Guo, Jiawei, et al.
Veröffentlicht: (2024)

Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
von: Sharma, Aman, et al.
Veröffentlicht: (2025)

JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
von: Cao, Jialun, et al.
Veröffentlicht: (2024)

Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks
von: Zhou, Yongxi, et al.
Veröffentlicht: (2026)

Assessing Large Language Models for Automated Feedback Generation in Learning Programming Problem Solving
von: Silva, Priscylla, et al.
Veröffentlicht: (2025)

Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy
von: Dong, Yihong, et al.
Veröffentlicht: (2026)

Can LLMs Reason Like Automated Theorem Provers for Rust Verification? VCoT-Bench: Evaluating via Verification Chain of Thought
von: Xie, Zichen, et al.
Veröffentlicht: (2026)

A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
von: Wu, Yixi, et al.
Veröffentlicht: (2024)

Analyzing Latent Concepts in Code Language Models
von: Sharma, Arushi, et al.
Veröffentlicht: (2025)

Redundancy and Concept Analysis for Code-trained Language Models
von: Sharma, Arushi, et al.
Veröffentlicht: (2023)

Generating Streamlining Constraints with Large Language Models
von: Voboril, Florentina, et al.
Veröffentlicht: (2024)

Are Large Language Models Memorizing Bug Benchmarks?
von: Ramos, Daniel, et al.
Veröffentlicht: (2024)

LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages
von: Diehl, Patrick, et al.
Veröffentlicht: (2025)

Benchmarking Large Language Models with Integer Sequence Generation Tasks
von: O'Malley, Daniel, et al.
Veröffentlicht: (2024)

Toward Explaining Large Language Models in Software Engineering Tasks
von: Vitale, Antonio, et al.
Veröffentlicht: (2025)

The Impact of Fine-tuning Large Language Models on Automated Program Repair
von: Macháček, Roman, et al.
Veröffentlicht: (2025)

REFLEX: Reference-Free Evaluation of Log Summarization via Large Language Model Judgment
von: Mudgal, Priyanka
Veröffentlicht: (2025)

The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute
von: Sharma, Aman, et al.
Veröffentlicht: (2025)

ProToken: Token-Level Attribution for Federated Large Language Models
von: Gill, Waris, et al.
Veröffentlicht: (2026)

Assessing the Impact of Code Changes on the Fault Localizability of Large Language Models
von: Haroon, Sabaat, et al.
Veröffentlicht: (2025)

Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes
von: Liu, Jingxiong, et al.
Veröffentlicht: (2024)

Effective Large Language Model Debugging with Best-first Tree Search
von: Song, Jialin, et al.
Veröffentlicht: (2024)

RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
von: Gautam, Dhruv, et al.
Veröffentlicht: (2025)

Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?
von: Wu, Guanghan, et al.
Veröffentlicht: (2025)

Zero-Shot Attribution for Large Language Models: A Distribution Testing Approach
von: Canonne, Clément L., et al.
Veröffentlicht: (2025)

AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks
von: Oulkadda, Ilyas, et al.
Veröffentlicht: (2025)

CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models
von: Zhang, Yifan, et al.
Veröffentlicht: (2025)

Large Language Models Should Ask Clarifying Questions to Increase Confidence in Generated Code
von: Wu, Jie JW
Veröffentlicht: (2023)

Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
von: Lin, Qiqiang, et al.
Veröffentlicht: (2024)

Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study
von: Storhaug, André, et al.
Veröffentlicht: (2024)

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
von: Di, Peng, et al.
Veröffentlicht: (2023)

MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation
von: Shbita, Basel, et al.
Veröffentlicht: (2025)

Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Models
von: Chen, Jie, et al.
Veröffentlicht: (2024)

Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localization
von: Rafi, Md Nakhla, et al.
Veröffentlicht: (2024)

StackSight: Unveiling WebAssembly through Large Language Models and Neurosymbolic Chain-of-Thought Decompilation
von: Fang, Weike, et al.
Veröffentlicht: (2024)