Cover Image

Are Large Language Models Robust in Understanding Code Against Semantics-Preserving Mutations?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Orvalho, Pedro, Kwiatkowska, Marta
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PyVeritas: On Verifying Python via LLM-Based Transpilation and Bounded Model Checking for C
by: Orvalho, Pedro, et al.
Published: (2025)

Model-Based Diagnosis with Multiple Observations: A Unified Approach for C Software and Boolean Circuits
by: Orvalho, Pedro, et al.
Published: (2025)

Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization
by: Orvalho, Pedro, et al.
Published: (2024)

An Empirical Study on Capability of Large Language Models in Understanding Code Semantics
by: Nguyen, Thu-Trang, et al.
Published: (2024)

CFaults: Model-Based Diagnosis for Fault Localization in C Programs with Multiple Test Cases
by: Orvalho, Pedro, et al.
Published: (2024)

Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
by: Pan, Zhiyuan, et al.
Published: (2025)

Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMs
by: Li, Ziyu, et al.
Published: (2024)

Auto-SPT: Automating Semantic Preserving Transformations for Code
by: Hooda, Ashish, et al.
Published: (2025)

Understanding Defects in Generated Codes by Language Models
by: Esfahani, Ali Mohammadi, et al.
Published: (2024)

Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models
by: Wang, Jian, et al.
Published: (2025)

InvAASTCluster: On Applying Invariant-Based Program Clustering to Introductory Programming Assignments
by: Orvalho, Pedro, et al.
Published: (2022)

Robustness and Reasoning Fidelity of Large Language Models in Long-Context Code Question Answering
by: Maharaj, Kishan, et al.
Published: (2026)

Testing the Effect of Code Documentation on Large Language Model Code Understanding
by: Macke, William, et al.
Published: (2024)

RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation
by: Lin, Feng, et al.
Published: (2025)

Evaluating Large Language Models for Code Review
by: Cihan, Umut, et al.
Published: (2025)

Task Abstention for Large Language Models in Code Generation
by: Zhou, Yanke, et al.
Published: (2026)

Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements
by: Abtahi, Seyed Moein, et al.
Published: (2025)

GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
by: Hou, Shuyang, et al.
Published: (2024)

CodeSSM: Towards State Space Models for Code Understanding
by: Verma, Shweta, et al.
Published: (2025)

Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications
by: Jin, Haolin, et al.
Published: (2025)

Large Language Model Guided Self-Debugging Code Generation
by: Adnan, Muntasir, et al.
Published: (2025)

Personality-Guided Code Generation Using Large Language Models
by: Guo, Yaoqi, et al.
Published: (2024)

Bugs in Large Language Models Generated Code: An Empirical Study
by: Tambon, Florian, et al.
Published: (2024)

Assessing the Code Clone Detection Capability of Large Language Models
by: Zhang, Zixian, et al.
Published: (2024)

CodeFort: Robust Training for Code Generation Models
by: Zhang, Yuhao, et al.
Published: (2024)

ReDef: Do Code Language Models Truly Understand Code Changes for Just-in-Time Software Defect Prediction?
by: Nam, Doha, et al.
Published: (2025)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
by: He, Mengliang, et al.
Published: (2025)

An Execution-Verified Multi-Language Benchmark for Code Semantic Reasoning
by: Li, Yikun, et al.
Published: (2026)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
by: Padwal, Vedant
Published: (2026)

Towards Leveraging Large Language Model Summaries for Topic Modeling in Source Code
by: Carissimi, Michele, et al.
Published: (2025)

Large Language Model-Driven Code Compliance Checking in Building Information Modeling
by: Madireddy, Soumya, et al.
Published: (2025)

Towards Privacy-Preserving Code Generation: Differentially Private Code Language Models
by: Catal, Melih, et al.
Published: (2025)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
by: Xiong, Zhanhang, et al.
Published: (2025)

Large Language Models in Code Co-generation for Safe Autonomous Vehicles
by: Nouri, Ali, et al.
Published: (2025)

Combining Large Language Models with Static Analyzers for Code Review Generation
by: Jaoua, Imen, et al.
Published: (2025)

Automated Code Review Using Large Language Models with Symbolic Reasoning
by: Icoz, Busra, et al.
Published: (2025)

CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
by: Gui, Ningxin, et al.
Published: (2025)

PerfCoder: Large Language Models for Interpretable Code Performance Optimization
by: Yang, Jiuding, et al.
Published: (2025)

Selecting and Combining Large Language Models for Scalable Code Clone Detection
by: Chochlov, Muslim, et al.
Published: (2025)

A Comparative Analysis of Large Language Models for Code Documentation Generation
by: Dvivedi, Shubhang Shekhar, et al.
Published: (2023)