Cover Image

ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Rao, Luo, Ziyang, Lin, Hongzhan, Ye, Zhen, Ma, Jing
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ScratchEval : A Multimodal Evaluation Framework for LLMs in Block-Based Programming
by: Si, Yuan, et al.
Published: (2026)

MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models
by: Wang, Shengkang, et al.
Published: (2024)

MM-CRITIC: A Holistic Evaluation of Large Multimodal Models as Multimodal Critique
by: Zeng, Gailun, et al.
Published: (2025)

MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models
by: Chen, Zixin, et al.
Published: (2025)

GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse
by: Lin, Hongzhan, et al.
Published: (2024)

CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models
by: Lin, Hongzhan, et al.
Published: (2024)

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
by: Zhao, Yuwei, et al.
Published: (2024)

AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness
by: Chen, Zixin, et al.
Published: (2025)

Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models
by: Lin, Hongzhan, et al.
Published: (2024)

Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking
by: Lin, Hongzhan, et al.
Published: (2026)

AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation
by: Luo, Ziyang, et al.
Published: (2024)

MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems
by: Li, Kaixin, et al.
Published: (2024)

DiffCoT: Diffusion-styled Chain-of-Thought Reasoning in LLMs
by: Cao, Shidong, et al.
Published: (2026)

MANBench: Is Your Multimodal Model Smarter than Human?
by: Zhou, Han, et al.
Published: (2025)

ClarityEthic: Explainable Moral Judgment Utilizing Contrastive Ethical Insights from Large Language Models
by: Sun, Yuxi, et al.
Published: (2024)

Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection
by: Yang, Ruichao, et al.
Published: (2026)

Towards Low-Resource Harmful Meme Detection with LMM Agents
by: Huang, Jianzhao, et al.
Published: (2024)

SHARP: Unlocking Interactive Hallucination via Stance Transfer in Role-Playing LLMs
by: Kong, Chuyi, et al.
Published: (2024)

LalaEval: A Holistic Human Evaluation Framework for Domain-Specific Large Language Models
by: Sun, Chongyan, et al.
Published: (2024)

VideoAutoArena: An Automated Arena for Evaluating Large Multimodal Models in Video Analysis through User Simulation
by: Luo, Ziyang, et al.
Published: (2024)

Reinforcement Tuning for Detecting Stances and Debunking Rumors Jointly with Large Language Models
by: Yang, Ruichao, et al.
Published: (2024)

ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use
by: Li, Kaixin, et al.
Published: (2025)

LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models
by: Zhang, Kaichen, et al.
Published: (2024)

LLM-Enhanced Multiple Instance Learning for Joint Rumor and Stance Detection with Social Context Information
by: Yang, Ruichao, et al.
Published: (2025)

MindScratch: A Visual Programming Support Tool for Classroom Learning Based on Multimodal Generative AI
by: Chen, Yunnong, et al.
Published: (2024)

From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms
by: Luo, Jinghao, et al.
Published: (2026)

FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models
by: Lin, Hongzhan, et al.
Published: (2025)

Explainable Ethical Assessment on Human Behaviors by Generating Conflicting Social Norms
by: Sun, Yuxi, et al.
Published: (2025)

"Is ChatGPT a Better Explainer than My Professor?": Evaluating the Explanation Capabilities of LLMs in Conversation Compared to a Human Baseline
by: Li, Grace, et al.
Published: (2024)

EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents
by: Cheng, Zhili, et al.
Published: (2025)

REFLEX: Self-Refining Explainable Fact-Checking via Verdict-Anchored Style Control
by: Kong, Chuyi, et al.
Published: (2025)

FormCoach: Lift Smarter, Not Harder
by: Zuo, Xiaoye, et al.
Published: (2025)

NeuroABench: A Multimodal Evaluation Benchmark for Neurosurgical Anatomy Identification
by: Song, Ziyang, et al.
Published: (2025)

GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation
by: Yan, Zhiyuan, et al.
Published: (2025)

Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom
by: Wang, Bo, et al.
Published: (2024)

FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation
by: He, Zheqi, et al.
Published: (2025)

FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions
by: Qin, Bowen, et al.
Published: (2025)

IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection
by: Li, Zewen, et al.
Published: (2025)

RoboSurg-VQA: A Multimodal Benchmark for Surgical Segmentation-Aware Visual Question Answering
by: Zhang, Chengyi, et al.
Published: (2026)

By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting
by: Yoon, Hyungjun, et al.
Published: (2024)