Cover Image

Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Jaeyeon, Shah, Kulin, Kontonis, Vasilis, Kakade, Sham, Chen, Sitan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training
by: Kim, Jaeyeon, et al.
Published: (2026)

Learning general Gaussian mixtures with efficient score matching
by: Chen, Sitan, et al.
Published: (2024)

Selective Underfitting in Diffusion Models
by: Song, Kiwhan, et al.
Published: (2025)

Any-Order Flexible Length Masked Diffusion
by: Kim, Jaeyeon, et al.
Published: (2025)

ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problems
by: Karan, Aayush, et al.
Published: (2025)

Masked Diffusion for Generative Recommendation
by: Shah, Kulin, et al.
Published: (2025)

Unrolled denoising networks provably learn optimal Bayesian inference
by: Karan, Aayush, et al.
Published: (2024)

Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning
by: Jin, Jikai, et al.
Published: (2025)

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
by: Zhang, Hanlin, et al.
Published: (2026)

Optimal Inference Schedules for Masked Diffusion Models
by: Chen, Sitan, et al.
Published: (2025)

Active Learning with Simple Questions
by: Kontonis, Vasilis, et al.
Published: (2024)

Learning Noisy Halfspaces with a Margin: Massart is No Harder than Random
by: Chandrasekaran, Gautam, et al.
Published: (2025)

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
by: Shah, Kulin, et al.
Published: (2025)

Characterization and Mitigation of Training Instabilities in Microscaling Formats
by: Su, Huangyuan, et al.
Published: (2025)

Online Learning of Halfspaces with Massart Noise
by: Diakonikolas, Ilias, et al.
Published: (2024)

Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension
by: Chandrasekaran, Gautam, et al.
Published: (2024)

GQ-VAE: A gated quantized VAE for learning variable length tokens
by: Datta, Theo, et al.
Published: (2025)

Wait, Wait, Wait... Why Do Reasoning Models Loop?
by: Pipis, Charilaos, et al.
Published: (2025)

Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent
by: Kou, Yiwen, et al.
Published: (2024)

LOTION: Smoothing the Optimization Landscape for Quantized Training
by: Kwun, Mujin, et al.
Published: (2025)

Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants
by: Morwani, Depen, et al.
Published: (2025)

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
by: Abreu, Natalie, et al.
Published: (2025)

Efficient Discrepancy Testing for Learning with Distribution Shift
by: Chandrasekaran, Gautam, et al.
Published: (2024)

Super Non-singular Decompositions of Polynomials and their Application to Robustly Learning Low-degree PTFs
by: Diakonikolas, Ilias, et al.
Published: (2024)

Understanding and Accelerating the Training of Masked Diffusion Language Models
by: Hong, Chunsan, et al.
Published: (2026)

Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models
by: Jelassi, Samy, et al.
Published: (2026)

Provably Learning Attention with Queries
by: Bhattamishra, Satwik, et al.
Published: (2026)

LoRA Training Provably Converges to a Low-Rank Global Minimum or It Fails Loudly (But it Probably Won't Fail)
by: Kim, Junsu, et al.
Published: (2025)

Universal Length Generalization with Turing Programs
by: Hou, Kaiying, et al.
Published: (2024)

Learning Hidden Markov Models Using Conditional Samples
by: Kakade, Sham M., et al.
Published: (2023)

Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond
by: Oncescu, Costin-Andrei, et al.
Published: (2024)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
by: Meterez, Alexandru, et al.
Published: (2025)

Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzles
by: Shah, Kulin, et al.
Published: (2024)

Simple Mechanisms for Representing, Indexing and Manipulating Concepts
by: Li, Yuanzhi, et al.
Published: (2023)

In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
by: Panigrahi, Abhishek, et al.
Published: (2025)

Deconstructing What Makes a Good Optimizer for Language Models
by: Zhao, Rosie, et al.
Published: (2024)

Repeat After Me: Transformers are Better than State Space Models at Copying
by: Jelassi, Samy, et al.
Published: (2024)

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
by: Li, Kenneth, et al.
Published: (2024)

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
by: Vyas, Nikhil, et al.
Published: (2023)

Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel
by: Gupta, Shivam, et al.
Published: (2024)