Cover Image

SOP-Maze: Evaluating Large Language Models on Complicated Business Standard Operating Procedures

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jiaming, Tang, Zhe, Jin, Zehao, Chen, Hefei, Jin, Yilin, Ding, Peng, Li, Xiaoyu, Cao, Xuezhi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability
by: wang, Jiaming, et al.
Published: (2025)

SocialMaze: A Benchmark for Evaluating Social Reasoning in Large Language Models
by: Xu, Zixiang, et al.
Published: (2025)

SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
by: Li, Zekun, et al.
Published: (2025)

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
by: Ding, Peng, et al.
Published: (2023)

UniHetero: Could Generation Enhance Understanding for Vision-Language-Model at Large Data Scale?
by: Chen, Fengjiao, et al.
Published: (2025)

ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents
by: Li, Zhigen, et al.
Published: (2024)

FM SO.P: A Progressive Task Mixture Framework with Automatic Evaluation for Cross-Domain SOP Understanding
by: Huang, Siyuan, et al.
Published: (2026)

AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO
by: Dao, Alan, et al.
Published: (2025)

Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Test
by: Pimenta, Rui A., et al.
Published: (2025)

Fusian: Multi-LoRA Fusion for Fine-Grained Continuous MBTI Personality Control in Large Language Models
by: Chen, Zehao, et al.
Published: (2026)

Datasets for Large Language Models: A Comprehensive Survey
by: Liu, Yang, et al.
Published: (2024)

TongGu: Mastering Classical Chinese Understanding with Knowledge-Grounded Large Language Models
by: Cao, Jiahuan, et al.
Published: (2024)

SDGO: Self-Discrimination-Guided Optimization for Consistent Safety in Large Language Models
by: Ding, Peng, et al.
Published: (2025)

Premise Order Matters in Reasoning with Large Language Models
by: Chen, Xinyun, et al.
Published: (2024)

Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models
by: Zhou, Yuqing, et al.
Published: (2024)

xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
by: Yu, Qingchen, et al.
Published: (2024)

DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models
by: Gao, Shangqian, et al.
Published: (2024)

Large Language Models as Tool Makers
by: Cai, Tianle, et al.
Published: (2023)

MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
by: Li, Zhiyu, et al.
Published: (2025)

C$^{3}$Bench: A Comprehensive Classical Chinese Understanding Benchmark for Large Language Models
by: Cao, Jiahuan, et al.
Published: (2024)

Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation
by: Yang, Yifan, et al.
Published: (2024)

Disentangling Language and Culture for Evaluating Multilingual Large Language Models
by: Ying, Jiahao, et al.
Published: (2025)

Exploring the Maze of Multilingual Modeling
by: Nezhad, Sina Bagheri, et al.
Published: (2023)

Stochastic Attention: Connectome-Inspired Randomized Routing for Expressive Linear-Time Attention
by: Jin, Zehao, et al.
Published: (2026)

Accounting Reasoning in Large Language Models: Concepts, Evaluation, and Empirical Analysis
by: Zhou, Jie, et al.
Published: (2025)

Can Large Language Models Generalize Procedures Across Representations?
by: Lin, Fangru, et al.
Published: (2026)

AMO-Bench: Large Language Models Still Struggle in High School Math Competitions
by: An, Shengnan, et al.
Published: (2025)

Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency
by: Fan, Siqi, et al.
Published: (2025)

Evaluating Accounting Reasoning Capabilities of Large Language Models
by: Zhou, Jie, et al.
Published: (2026)

Beyond Benchmarking: A New Paradigm for Evaluation and Assessment of Large Language Models
by: Liu, Jin, et al.
Published: (2024)

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation
by: Wang, Xinyi, et al.
Published: (2026)

One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models
by: Cao, Pengfei, et al.
Published: (2024)

DABL: Detecting Semantic Anomalies in Business Processes Using Large Language Models
by: Guan, Wei, et al.
Published: (2024)

Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models
by: Ying, Jiahao, et al.
Published: (2024)

ProBench: Benchmarking Large Language Models in Competitive Programming
by: Yang, Lei, et al.
Published: (2025)

Evaluating Large Language Models on the GMAT: Implications for the Future of Business Education
by: Ashrafimoghari, Vahid, et al.
Published: (2024)

EMNLP: Educator-role Moral and Normative Large Language Models Profiling
by: Jiang, Yilin, et al.
Published: (2025)

Large Language Models as Optimizers
by: Yang, Chengrun, et al.
Published: (2023)

StoryBox: Collaborative Multi-Agent Simulation for Hybrid Bottom-Up Long-Form Story Generation Using Large Language Models
by: Chen, Zehao, et al.
Published: (2025)

OWL: A Large Language Model for IT Operations
by: Guo, Hongcheng, et al.
Published: (2023)