Cover Image

MatheMagic: Generating Dynamic Mathematics Benchmarks Robust to Memorization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	O'Brien, Dayyán, Haddow, Barry, Allaway, Emily, Chen, Pinzhen
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Generics are puzzling. Can language models find the missing piece?
by: Calderón, Gustavo Cilleruelo, et al.
Published: (2024)

DocHPLT: A Massively Multilingual Document-Level Translation Dataset
by: O'Brien, Dayyán, et al.
Published: (2025)

Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models?
by: Chen, Pinzhen, et al.
Published: (2024)

Iterative Translation Refinement with Large Language Models
by: Chen, Pinzhen, et al.
Published: (2023)

Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca
by: Chen, Pinzhen, et al.
Published: (2023)

MGen: Millions of Naturally Occurring Generics in Context
by: Cilleruelo, Gustavo, et al.
Published: (2025)

Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task
by: Ranaldi, Leonardo, et al.
Published: (2025)

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models
by: Ji, Shaoxiong, et al.
Published: (2024)

Context and System Fusion in Post-ASR Emotion Recognition with Large Language Models
by: Stepachev, Pavel, et al.
Published: (2024)

RoMath: A Mathematical Reasoning Benchmark in Romanian
by: Cosma, Adrian, et al.
Published: (2024)

DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models
by: Zou, Chengke, et al.
Published: (2024)

MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
by: Fang, Meng, et al.
Published: (2024)

The Ups and Downs of Large Language Model Inference with Vocabulary Trimming by Language Heuristics
by: Bogoychev, Nikolay, et al.
Published: (2023)

TeleMath: A Benchmark for Large Language Models in Telecom Mathematical Problem Solving
by: Colle, Vincenzo, et al.
Published: (2025)

ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models
by: Wu, Yanan, et al.
Published: (2024)

FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
by: Liu, Yan, et al.
Published: (2024)

Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
by: Prashanth, USVSN Sai, et al.
Published: (2024)

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation
by: Iyer, Vivek, et al.
Published: (2025)

MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs
by: Lu, Zimu, et al.
Published: (2024)

VisAidMath: Benchmarking Visual-Aided Mathematical Reasoning
by: Ma, Jingkun, et al.
Published: (2024)

MathAgent: Adversarial Evolution of Constraint Graphs for Mathematical Reasoning Data Synthesis
by: Yu, Zixiong, et al.
Published: (2026)

DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving
by: Tong, Yuxuan, et al.
Published: (2024)

MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Models
by: Peng, Shuai, et al.
Published: (2024)

Context Memorization for Efficient Long Context Generation
by: Okoshi, Yasuyuki, et al.
Published: (2026)

SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese
by: Xu, Liang, et al.
Published: (2024)

MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model
by: Yang, Zhen, et al.
Published: (2024)

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
by: Zhu, Wenhao, et al.
Published: (2025)

DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
by: Shao, Zhihong, et al.
Published: (2025)

MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion
by: Pei, Qizhi, et al.
Published: (2025)

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
by: Yu, Longhui, et al.
Published: (2023)

MathSpeech: Leveraging Small LMs for Accurate Conversion in Mathematical Speech-to-Formula
by: Hyeon, Sieun, et al.
Published: (2024)

Pitfalls and Outlooks in Using COMET
by: Zouhar, Vilém, et al.
Published: (2024)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

MathScale: Scaling Instruction Tuning for Mathematical Reasoning
by: Tang, Zhengyang, et al.
Published: (2024)

An Investigation of Robustness of LLMs in Mathematical Reasoning: Benchmarking with Mathematically-Equivalent Transformation of Advanced Mathematical Problems
by: Hao, Yuren, et al.
Published: (2025)

AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation
by: Liu, Xianyang, et al.
Published: (2025)

ControlMath: Controllable Data Generation Promotes Math Generalist Models
by: Chen, Nuo, et al.
Published: (2024)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
by: Xiao, Jianfei, et al.
Published: (2026)

Adversarial Math Word Problem Generation
by: Xie, Roy, et al.
Published: (2024)

MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task
by: Yan, Yuchen, et al.
Published: (2025)