Cover Image

LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Shulin, Ma, Shirong, Li, Yinghui, Huang, Mengzuo, Zou, Wuhe, Zhang, Weidong, Zheng, Hai-Tao
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HeLM: Highlighted Evidence augmented Language Model for Enhanced Table-to-Text Generation
by: Bian, Junyi, et al.
Published: (2023)

From Retrieval to Generation: Efficient and Effective Entity Set Expansion
by: Huang, Shulin, et al.
Published: (2023)

Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correction
by: Li, Yinghui, et al.
Published: (2023)

When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models
by: Li, Yinghui, et al.
Published: (2024)

Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles
by: Chen, Qi, et al.
Published: (2024)

Evaluation Hallucination in Multi-Round Incomplete Information Lateral-Driven Reasoning Tasks
by: Dong, Wenhan, et al.
Published: (2025)

AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzles
by: Panagiotopoulos, Ioannis, et al.
Published: (2024)

On the (In)Effectiveness of Large Language Models for Chinese Text Correction
by: Li, Yinghui, et al.
Published: (2023)

Missed Connections: Lateral Thinking Puzzles for Large Language Models
by: Todd, Graham, et al.
Published: (2024)

MCEval: A Dynamic Framework for Fair Multilingual Cultural Evaluation of LLMs
by: Huang, Shulin, et al.
Published: (2025)

Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework
by: Xing, Peng, et al.
Published: (2024)

Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark
by: Xu, Zhikun, et al.
Published: (2024)

BotEval: Facilitating Interactive Human Evaluation
by: Cho, Hyundong, et al.
Published: (2024)

Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs
by: Badola, Kartikeya, et al.
Published: (2025)

DependEval: Benchmarking LLMs for Repository Dependency Understanding
by: Du, Junjia, et al.
Published: (2025)

OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety
by: Liu, Chuang, et al.
Published: (2024)

TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning
by: Liu, Daixian, et al.
Published: (2026)

Automatic Context Pattern Generation for Entity Set Expansion
by: Li, Yinghui, et al.
Published: (2022)

MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages
by: Zhang, Chen, et al.
Published: (2025)

MM-Eval: A Hierarchical Benchmark for Modern Mongolian Evaluation in LLMs
by: Zhang, Mengyuan, et al.
Published: (2024)

CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generation
by: Leng, Jixuan, et al.
Published: (2025)

HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMs
by: Cheng, Tsz Chung, et al.
Published: (2025)

PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions
by: Niazi, Ruhallah, et al.
Published: (2026)

RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
by: Huang, Zhongzhan, et al.
Published: (2025)

CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models
by: Yu, Linhao, et al.
Published: (2024)

Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities
by: Kuang, Jiayi, et al.
Published: (2025)

CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models
by: Qiu, Zexuan, et al.
Published: (2024)

ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions
by: Ye, Jingheng, et al.
Published: (2024)

GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving
by: Zhang, Jiaxin, et al.
Published: (2024)

StackEval: Benchmarking LLMs in Coding Assistance
by: Shah, Nidhish, et al.
Published: (2024)

Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
by: Huang, Shulin, et al.
Published: (2025)

Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent
by: Li, Yangning, et al.
Published: (2024)

Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
by: Li, Yangning, et al.
Published: (2025)

Fusion-Eval: Integrating Assistant Evaluators with LLMs
by: Shu, Lei, et al.
Published: (2023)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
by: Yang, Jialin, et al.
Published: (2025)

ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning
by: Huang, Shulin, et al.
Published: (2025)

PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles
by: Long, Yitao, et al.
Published: (2025)

Are LLMs Effective Backbones for Fine-tuning? An Experimental Investigation of Supervised LLMs on Chinese Short Text Matching
by: Liu, Shulin, et al.
Published: (2024)

StressEval: Failure-Driven Dynamic Benchmarking for Knowledge-Intensive Reasoning in Large Language Models
by: Chen, Yongrui, et al.
Published: (2026)

Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMs
by: Mendonça, John, et al.
Published: (2024)