Cover Image

MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Juneja, Gurusha, Albalak, Alon, Hua, Wenyue, Wang, William Yang
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
by: Juneja, Gurusha, et al.
Published: (2025)

$\texttt{LM}^\texttt{2}$: A Simple Society of Language Models Solves Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2024)

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2023)

Task Facet Learning: A Structured Approach to Prompt Optimization
by: Juneja, Gurusha, et al.
Published: (2024)

Adversarial Training for Process Reward Models
by: Juneja, Gurusha, et al.
Published: (2025)

Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable
by: Saha, Rounak, et al.
Published: (2026)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
by: Wang, Xinyi, et al.
Published: (2024)

ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models
by: Zhao, Zhixue, et al.
Published: (2024)

MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate
by: Amayuelas, Alfonso, et al.
Published: (2024)

MoralBench: Moral Evaluation of LLMs
by: Ji, Jianchao, et al.
Published: (2024)

RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
by: Zhou, Ruiwen, et al.
Published: (2024)

SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
by: Li, Zekun, et al.
Published: (2025)

REALM: A Dataset of Real-World LLM Use Cases
by: Cheng, Jingwen, et al.
Published: (2025)

InductionBench: LLMs Fail in the Simplest Complexity Class
by: Hua, Wenyue, et al.
Published: (2025)

Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
by: Albalak, Alon, et al.
Published: (2025)

War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars
by: Hua, Wenyue, et al.
Published: (2023)

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
by: Xiang, Violet, et al.
Published: (2025)

Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents
by: Li, Zelong, et al.
Published: (2024)

Turbulence-like 5/3 spectral scaling in contextual representations of language as a complex system
by: Yang, Zhongxin, et al.
Published: (2026)

Simulated Reasoning is Reasoning
by: Kempt, Hendrik, et al.
Published: (2026)

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
by: Fan, Lizhou, et al.
Published: (2023)

The Impact of Reasoning Step Length on Large Language Models
by: Jin, Mingyu, et al.
Published: (2024)

Failure of contextual invariance in large language models
by: Kumar, Sagar, et al.
Published: (2026)

Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks
by: Hua, Wenyue, et al.
Published: (2024)

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
by: Levy, Mosh, et al.
Published: (2024)

Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance
by: Alon, Bar, et al.
Published: (2026)

A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
by: Zhang, Qiyuan, et al.
Published: (2025)

CareMedEval dataset: Evaluating Critical Appraisal and Reasoning in the Biomedical Field
by: Bonzi, Doria, et al.
Published: (2025)

UP5: Unbiased Foundation Model for Fairness-aware Recommendation
by: Hua, Wenyue, et al.
Published: (2023)

Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical texts
by: Kumar, Aditya, et al.
Published: (2025)

VIDA: A dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
by: Pan, Jingheng, et al.
Published: (2026)

Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems
by: Zhou, Ruiwen, et al.
Published: (2026)

Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense
by: Ouyang, Yang, et al.
Published: (2025)

Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions
by: Barve, Saharsh, et al.
Published: (2025)

A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interaction
by: Pan, Ruihao, et al.
Published: (2026)

TrustAgent: Towards Safe and Trustworthy LLM-based Agents
by: Hua, Wenyue, et al.
Published: (2024)

A Multi-To-One Interview Paradigm for Efficient MLLM Evaluation
by: Shen, Ye, et al.
Published: (2025)

AutoFlow: Automated Workflow Generation for Large Language Model Agents
by: Li, Zelong, et al.
Published: (2024)

MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models
by: Ye, Junjie, et al.
Published: (2025)

A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations
by: Tan, Andong, et al.
Published: (2026)