Image de couverture de livre

MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dai, Jianbo, Lu, Jianqiao, Feng, Yunlong, Zeng, Guangtao, Ruan, Rongju, Cheng, Ming, Huang, Dong, Tan, Haochen, Guo, Zhijiang
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning
par: Huang, Dong, et autres
Publié: (2024)

AutoPSV: Automated Process-Supervised Verifier
par: Lu, Jianqiao, et autres
Publié: (2024)

PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models
par: Tan, Haochen, et autres
Publié: (2024)

Humanity's Last Code Exam: Can Advanced LLMs Conquer Human's Hardest Code Competition?
par: Li, Xiangyang, et autres
Publié: (2025)

EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization
par: Huang, Dong, et autres
Publié: (2024)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
par: Chen, Zaoyu, et autres
Publié: (2026)

FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
par: Wan, Yingjia, et autres
Publié: (2025)

FormalAlign: Automated Alignment Evaluation for Autoformalization
par: Lu, Jianqiao, et autres
Publié: (2024)

CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts
par: Zeng, Qingkai, et autres
Publié: (2024)

MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs
par: Zeng, Zhongshen, et autres
Publié: (2024)

Quasinormal Modes of pp-Wave Spacetimes and Zero Temperature Dissipation
par: Dai, Huayu, et autres
Publié: (2026)

Exploring the Capabilities and Limitations of Large Language Models in the Electric Energy Sector
par: Majumder, Subir, et autres
Publié: (2024)

Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models
par: Huang, Beichen, et autres
Publié: (2024)

Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration
par: Dai, Sunhao, et autres
Publié: (2024)

Language Models and Cycle Consistency for Self-Reflective Machine Translation
par: Wangni, Jianqiao
Publié: (2024)

Beyond Accuracy: Characterizing Code Comprehension Capabilities in (Large) Language Models
par: Mächtle, Felix, et autres
Publié: (2026)

Beyond Information: Mechanisms of Risk Communication on Self‐Protective Behavior in Public Health Emergencies
par: Yunpeng Xu, et autres
Publié: (2026)

Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning
par: Yang, Rem, et autres
Publié: (2025)

Do We Need Language-Specific Fact-Checking Models? The Case of Chinese
par: Zhang, Caiqi, et autres
Publié: (2024)

AEGIS: Exploring the Limit of World Knowledge Capabilities for Unified Mulitmodal Models
par: Lin, Jintao, et autres
Publié: (2026)

TinyLlama: An Open-Source Small Language Model
par: Zhang, Peiyuan, et autres
Publié: (2024)

Beyond Static Evaluation: A Dynamic Approach to Assessing AI Assistants' API Invocation Capabilities
par: Mu, Honglin, et autres
Publié: (2024)

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
par: Wu, Zhaofeng, et autres
Publié: (2023)

Exploring the Capabilities and Limitations of Large Language Models for Radiation Oncology Decision Support
par: Putz, Florian, et autres
Publié: (2025)

Are LLMs Rigorous Logical Reasoners? Empowering Natural Language Proof Generation by Stepwise Decoding with Contrastive Learning
par: Su, Ying, et autres
Publié: (2023)

TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios
par: Wei, Shaohang, et autres
Publié: (2025)

Improving Numerical Error Bounds Near Sharp Interface Limit for Stochastic Reaction-Diffusion Equations
par: Cui, Jianbo, et autres
Publié: (2024)

Deepfake Detection in Image Sequences: A Temporal Approach for Anomaly Detection
par: Rongju Yao, et autres
Publié: (2025)

Exploring and Benchmarking the Planning Capabilities of Large Language Models
par: Bohnet, Bernd, et autres
Publié: (2024)

SailCompass: Towards Reproducible and Robust Evaluation for Southeast Asian Languages
par: Guo, Jia, et autres
Publié: (2024)

Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search
par: Li, Haochen, et autres
Publié: (2024)

Measuring the Oscillation Frequency Beyond the Diffraction Limit
par: Hu, Chao-Ning, et autres
Publié: (2025)

CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation
par: Yan, Kaiwen, et autres
Publié: (2025)

YODA: Teacher-Student Progressive Learning for Language Models
par: Lu, Jianqiao, et autres
Publié: (2024)

CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
par: Guo, Jiawei, et autres
Publié: (2024)

On the Limitation of Kernel Dependence Maximization for Feature Selection
par: Liu, Keli, et autres
Publié: (2024)

Exploring the Capabilities of LLMs for Code Change Related Tasks
par: Fan, Lishui, et autres
Publié: (2024)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
par: He, Mengliang, et autres
Publié: (2025)

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
par: Yao, Yuxuan, et autres
Publié: (2026)

CiteCheck: Towards Accurate Citation Faithfulness Detection
par: Xu, Ziyao, et autres
Publié: (2025)