Cover Image

StableMask: Refining Causal Masking in Decoder-only Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yin, Qingyu, He, Xuzheng, Zhuang, Xiang, Zhao, Yu, Yao, Jianhua, Shen, Xiaoyu, Zhang, Qiang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
by: Yao, Lin
Published: (2026)

Empirical Analysis of Decoding Biases in Masked Diffusion Models
by: Huang, Pengcheng, et al.
Published: (2025)

Intermittent Semi-Working Mask: A New Masking Paradigm for LLMs
by: Hu, HaoYuan, et al.
Published: (2024)

Exploration of Masked and Causal Language Modelling for Text Generation
by: Micheletti, Nicolo, et al.
Published: (2024)

Entropy-aware Masking for Masked Language Modeling
by: Srinivasagan, Gokul, et al.
Published: (2026)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
by: Liang, Wen, et al.
Published: (2024)

Chinese ModernBERT with Whole-Word Masking
by: Zhao, Zeyu, et al.
Published: (2025)

Token Masking Improves Transformer-Based Text Classification
by: Xu, Xianglong, et al.
Published: (2025)

MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
by: Deng, Jingyuan, et al.
Published: (2025)

Variational Masked Diffusion Models
by: Zhang, Yichi, et al.
Published: (2025)

Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
by: Chen, Ziyu, et al.
Published: (2025)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
by: Lin, Ailiang, et al.
Published: (2025)

Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
by: Zheng, Kaiwen, et al.
Published: (2024)

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
by: Zheng, Bo, et al.
Published: (2026)

Mask-guided BERT for Few Shot Text Classification
by: Liao, Wenxiong, et al.
Published: (2023)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step
by: Yang, Jingyi, et al.
Published: (2025)

SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
by: Wang, Chenyu, et al.
Published: (2025)

Inconsistencies in Masked Language Models
by: Young, Tom, et al.
Published: (2022)

RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts
by: Yang, Jing, et al.
Published: (2024)

Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models
by: Zhong, Linhao, et al.
Published: (2026)

Masked Diffusion Models as Energy Minimization
by: Chen, Sitong, et al.
Published: (2025)

How Do Humans Write Code? Large Models Do It the Same Way Too
by: Li, Long, et al.
Published: (2024)

Memorization in Attention-only Transformers
by: Dana, Léo, et al.
Published: (2024)

Scaling up Masked Diffusion Models on Text
by: Nie, Shen, et al.
Published: (2024)

Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
by: Chen, Qian, et al.
Published: (2023)

Dynamic Masking Rate Schedules for MLM Pretraining
by: Ankner, Zachary, et al.
Published: (2023)

Infinite Mask Diffusion for Few-Step Distillation
by: Yoo, Jaehoon, et al.
Published: (2026)

Defensive Dual Masking for Robust Adversarial Defense
by: Yang, Wangli, et al.
Published: (2024)

Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling
by: Kesgin, Himmet Toprak, et al.
Published: (2024)

Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models
by: Wang, Shuxun, et al.
Published: (2025)

MiSS: Revisiting the Trade-off in LoRA with an Efficient Shard-Sharing Structure
by: Kang, Jiale, et al.
Published: (2024)

Alignment-Enhanced Decoding:Defending via Token-Level Adaptive Refining of Probability Distributions
by: Liu, Quan, et al.
Published: (2024)

Time-Aware Feature Selection: Adaptive Temporal Masking for Stable Sparse Autoencoder Training
by: Li, T. Ed, et al.
Published: (2025)

ShardMemo: Masked MoE Routing for Sharded Agentic LLM Memory
by: Zhao, Yang, et al.
Published: (2026)

Probing the Difficulty Perception Mechanism of Large Language Models
by: Lee, Sunbowen, et al.
Published: (2025)

Reflection-Window Decoding: Text Generation with Selective Refinement
by: Tang, Zeyu, et al.
Published: (2025)

Debiasing LLMs by Masking Unfairness-Driving Attention Heads
by: Han, Tingxu, et al.
Published: (2025)

EMBRE: Entity-aware Masking for Biomedical Relation Extraction
by: Li, Mingjie, et al.
Published: (2024)

DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
by: Yu, Longxuan, et al.
Published: (2026)