Cover Image

CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Manh, Dung Nguyen, Chau, Thang Phan, Hai, Nam Le, Doan, Thong T., Nguyen, Nam V., Pham, Quang, Bui, Nghi D. Q.
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Impacts of Contexts on Repository-Level Code Generation
by: Hai, Nam Le, et al.
Published: (2024)

CollabCoder: Plan-Code Co-Evolution via Collaborative Decision-Making for Efficient Code Generation
by: Doan, Duy Tung, et al.
Published: (2026)

HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale
by: Phan, Huy Nhat, et al.
Published: (2024)

On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating
by: Nguyen, Huy, et al.
Published: (2025)

Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond
by: Le-Anh, Minh, et al.
Published: (2026)

LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models
by: Nguyen, Nam V., et al.
Published: (2024)

AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology
by: Nguyen, Minh Huynh, et al.
Published: (2024)

RepoHyper: Search-Expand-Refine on Semantic Graphs for Repository-Level Code Completion
by: Phan, Huy N., et al.
Published: (2024)

Functional Overlap Reranking for Neural Code Generation
by: To, Hung Quoc, et al.
Published: (2023)

Mastering the Craft of Data Synthesis for CodeLLMs
by: Chen, Meng, et al.
Published: (2024)

Envisioning the Next-Generation AI Coding Assistants: Insights & Proposals
by: Nghiem, Khanh, et al.
Published: (2024)

VisualCoder: Guiding Large Language Models in Code Execution with Fine-grained Multimodal Chain-of-Thought Reasoning
by: Le, Cuong Chi, et al.
Published: (2024)

CodeFlow: Program Behavior Prediction with Dynamic Dependencies Learning
by: Le, Cuong Chi, et al.
Published: (2024)

Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned
by: Bui, Nghi D. Q.
Published: (2026)

Dopamin: Transformer-based Comment Classifiers through Domain Post-Training and Multi-level Layer Aggregation
by: Hai, Nam Le, et al.
Published: (2024)

Agentic Coding Needs Proactivity, Not Just Autonomy
by: Bui, Nghi D. Q., et al.
Published: (2026)

Evaluating and Aligning CodeLLMs on Human Preference
by: Yang, Jian, et al.
Published: (2024)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

Spec-TOD: A Specialized Instruction-Tuned LLM Framework for Efficient Task-Oriented Dialogue Systems
by: Nguyen, Quang-Vinh, et al.
Published: (2025)

Understanding How CodeLLMs (Mis)Predict Types with Activation Steering
by: Lucchetti, Francesca, et al.
Published: (2024)

Detection of Technical Debt in Java Source Code
by: Hai, Nam Le, et al.
Published: (2024)

CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases
by: Hoang, Anh Nguyen, et al.
Published: (2025)

Context-Aware CodeLLM Eviction for AI-assisted Coding
by: Thangarajah, Kishanthan, et al.
Published: (2025)

Planning Optimal Trajectories for Mobile Manipulators under End-effector Trajectory Continuity Constraint
by: Nguyen, Quang-Nam, et al.
Published: (2023)

CodeTF: One-stop Transformer Library for State-of-the-art Code LLMs
by: Bui, Nghi D. Q., et al.
Published: (2023)

SynConfRoute: Syntax-Aware Routing for Efficient Code Completion with Small CodeLLMs
by: Thangarajah, Kishanthan, et al.
Published: (2026)

DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies
by: Dau, Anh T. V., et al.
Published: (2023)

When Names Disappear: Revealing What LLMs Actually Understand About Code
by: Le, Cuong Chi, et al.
Published: (2025)

Localizing Malicious Outputs from CodeLLM
by: Borana, Mayukh, et al.
Published: (2025)

Motion Free B-frame Coding for Neural Video Compression
by: Nguyen, Van Thang
Published: (2024)

Climatological regime and weather condition occurred on the cruise expedition (May 1999) on Vietnam continental shelf
by: Thong, Bui Xuan, et al.
Published: (2001)

COBOLAssist: Analyzing and Fixing Compilation Errors for LLM-Powered COBOL Code Generation
by: Dau, Anh T. V., et al.
Published: (2026)

Some species of macro-gastropods in the coastal zone of Khanh Hoa province
by: Bui, Quang Nghi
Published: (2020)

New record of the Ophuroid Ophiosphaera insignis Brock, 1888 (Ophiuroidea- Echinodermata) in Vietnamese seawaters.
by: Nguyen, Thi My Ngan, et al.
Published: (2015)

Description of Stichopus sp. (phylum Echinodermata – class Holothuroidea) collected in Nha Trang bay
by: Nguyen, Thi My Ngan, et al.
Published: (2015)

Development of primer‐introduced restriction analysis PCR for detecting polymorphism of two cis‐regulatory SNPs upstream of ABCG2 conferring blue eggshell trait
by: Anh Phu Nam Bui, et al.
Published: (2025)

DMLDroid: Deep Multimodal Fusion Framework for Android Malware Detection with Resilience to Code Obfuscation and Adversarial Perturbations
by: Trung, Doan Minh, et al.
Published: (2025)

COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation
by: Dau, Anh T. V., et al.
Published: (2026)

CoReTab: Improving Multimodal Table Understanding with Code-driven Reasoning
by: Nguyen, Van-Quang, et al.
Published: (2026)

Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection
by: Nguyen, Dang-Khoa, et al.
Published: (2025)