Cover Image

TaskEval: Synthesised Evaluation for Foundation-Model Tasks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Widanapathiranage, Dilani, Barnett, Scott, Kurniawan, Stefanus, Takerngsaksiri, Wannita
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models
by: Tambon, Florian, et al.
Published: (2024)

Seven Failure Points When Engineering a Retrieval Augmented Generation System
by: Barnett, Scott, et al.
Published: (2024)

LLMs for Test Input Generation for Semantic Caches
by: Rasool, Zafaryab, et al.
Published: (2024)

Code Readability in the Age of Large Language Models: An Industrial Case Study from Atlassian
by: Takerngsaksiri, Wannita, et al.
Published: (2025)

Enhancing Large Language Models for Text-to-Testcase Generation
by: Alagarsamy, Saranya, et al.
Published: (2024)

PyTester: Deep Reinforcement Learning for Text-to-Testcase Generation
by: Takerngsaksiri, Wannita, et al.
Published: (2024)

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
by: Lu, Qinghua, et al.
Published: (2025)

Evaluating LLMs for Visualization Tasks
by: Khan, Saadiq Rauf, et al.
Published: (2025)

WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks
by: Wornow, Michael, et al.
Published: (2024)

Students' Perspective on AI Code Completion: Benefits and Challenges
by: Takerngsaksiri, Wannita, et al.
Published: (2023)

Human-In-the-Loop Software Development Agents
by: Takerngsaksiri, Wannita, et al.
Published: (2024)

Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers
by: Hassan, Ahmed E., et al.
Published: (2024)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
by: Zhang, Tanghaoran, et al.
Published: (2026)

The Foundations of Computational Management: A Systematic Approach to Task Automation for the Integration of Artificial Intelligence into Existing Workflows
by: Jadad-Garcia, Tamen, et al.
Published: (2024)

RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models
by: Liu, Jingjing, et al.
Published: (2025)

When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions
by: Larbi, Maya, et al.
Published: (2025)

TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
by: Bruches, Elena, et al.
Published: (2026)

Task Abstention for Large Language Models in Code Generation
by: Zhou, Yanke, et al.
Published: (2026)

Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation
by: Hamedi, Parisa, et al.
Published: (2025)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2024)

Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution
by: Shastry, KN Ajay, et al.
Published: (2026)

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents
by: Hong, Sirui, et al.
Published: (2026)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

EvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
by: Wen, Xin-Cheng, et al.
Published: (2024)

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
by: Ni, Ziyi, et al.
Published: (2025)

Spreadsheet Modeling Experiments Using GPTs on Small Problem Statements and the Wall Task
by: Grossman, Thomas A., et al.
Published: (2026)

Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling
by: Kavathekar, Ishan, et al.
Published: (2025)

Agentic Frameworks for Reasoning Tasks: An Empirical Study
by: Rasheed, Zeeshan, et al.
Published: (2026)

Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Tasks
by: Al-Kaswan, Ali, et al.
Published: (2025)

GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
by: Hou, Shuyang, et al.
Published: (2024)

Ambiguity Resolution with Human Feedback for Code Writing Tasks
by: Nandan, Aditey, et al.
Published: (2025)

An Empirical Study of Knowledge Distillation for Code Understanding Tasks
by: Wang, Ruiqi, et al.
Published: (2025)

Automated Benchmark Generation for Repository-Level Coding Tasks
by: Vergopoulos, Konstantinos, et al.
Published: (2025)

Task-oriented Prompt Enhancement via Script Generation
by: Wang, Chung-Yu, et al.
Published: (2024)

Multi-Task Program Error Repair and Explanatory Diagnosis
by: Xu, Zhenyu, et al.
Published: (2024)

Runtime-Structured Task Decomposition for Agentic Coding Systems
by: Asthana, Shubhi, et al.
Published: (2026)

AutoDroid: LLM-powered Task Automation in Android
by: Wen, Hao, et al.
Published: (2023)

CABENCH: Benchmarking Composable AI for Solving Complex Tasks through Composing Ready-to-Use Models
by: Pham, Tung-Thuy, et al.
Published: (2025)

Beyond Output Correctness: Benchmarking and Evaluating Large Language Model Reasoning in Coding Tasks
by: Li, Yuangang, et al.
Published: (2026)

Squeez: Task-Conditioned Tool-Output Pruning for Coding Agents
by: Kovács, Ádám
Published: (2026)