Cover Image

AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Qin, Huang, Fei, Peng, Runyu, Lu, Keming, Yu, Bowen, Cheng, Qinyuan, Qiu, Xipeng, Huang, Xuanjing, Lin, Junyang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation
by: Zhu, Qin, et al.
Published: (2024)

Scaling Laws for Fact Memorization of Large Language Models
by: Lu, Xingyu, et al.
Published: (2024)

Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models
by: Patel, Nisarg, et al.
Published: (2024)

Dynamic and Generalizable Process Reward Modeling
by: Yin, Zhangyue, et al.
Published: (2025)

AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems
by: Wang, Weiyi, et al.
Published: (2026)

Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model
by: Wang, Siyin, et al.
Published: (2024)

Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models
by: Xu, Ningyu, et al.
Published: (2026)

ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models
by: Yin, Zhangyue, et al.
Published: (2025)

Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information
by: Huang, Youcheng, et al.
Published: (2025)

F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods
by: Sun, Yu, et al.
Published: (2024)

LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models
by: Wan, Yuxuan, et al.
Published: (2024)

Evaluating the Logical Reasoning Abilities of Large Reasoning Models
by: Liu, Hanmeng, et al.
Published: (2025)

Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework
by: Sun, Yuhong, et al.
Published: (2025)

Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models
by: Liu, Peiju, et al.
Published: (2026)

MARGE: Improving Math Reasoning for LLMs with Guided Exploration
by: Gao, Jingyue, et al.
Published: (2025)

How to Set the Batch Size for Large-Scale Pre-training?
by: Zhou, Yunhua, et al.
Published: (2026)

Agent Alignment in Evolving Social Norms
by: Li, Shimin, et al.
Published: (2024)

LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
by: Liu, Yiwei, et al.
Published: (2025)

LLM can Achieve Self-Regulation via Hyperparameter Aware Generation
by: Wang, Siyin, et al.
Published: (2024)

Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
by: Zeng, Zhiyuan, et al.
Published: (2024)

Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews
by: Li, Bowen, et al.
Published: (2026)

LogiAgent: Automated Logical Testing for REST Systems with LLM-Based Multi-Agents
by: Zhang, Ke, et al.
Published: (2025)

VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search
by: Wang, Yikun, et al.
Published: (2025)

LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning
by: Zheng, Tianshi, et al.
Published: (2025)

Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metric
by: Cao, Yixin, et al.
Published: (2025)

Multi-hop Reasoning via Early Knowledge Alignment
by: Wang, Yuxin, et al.
Published: (2025)

How to Mitigate Overfitting in Weak-to-strong Generalization?
by: Shi, Junhao, et al.
Published: (2025)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model
by: Yuan, Chenhan, et al.
Published: (2024)

LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
by: Gui, Jiayi, et al.
Published: (2024)

Self-Evolving Critique Abilities in Large Language Models
by: Tang, Zhengyang, et al.
Published: (2025)

CodecBench: A Comprehensive Benchmark for Acoustic and Semantic Evaluation
by: Deng, Ruifan, et al.
Published: (2025)

LogiPlan: A Structured Benchmark for Logical Planning and Relational Reasoning in LLMs
by: Cai, Yanan, et al.
Published: (2025)

LogiDebrief: A Signal-Temporal Logic based Automated Debriefing Approach with Large Language Models Integration
by: Chen, Zirong, et al.
Published: (2025)

LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
by: Parmar, Mihir, et al.
Published: (2024)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
by: Wang, Bo, et al.
Published: (2025)

Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?
by: Zeng, Zhiyuan, et al.
Published: (2025)

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
by: Yin, Zhangyue, et al.
Published: (2024)

Data-free Weight Compress and Denoise for Large Language Models
by: Peng, Runyu, et al.
Published: (2024)

HardcoreLogic: Challenging Large Reasoning Models with Long-tail Logic Puzzle Games
by: Liang, Jingcong, et al.
Published: (2025)