Image de couverture de livre

StableMask: Refining Causal Masking in Decoder-only Transformer

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yin, Qingyu, He, Xuzheng, Zhuang, Xiang, Zhao, Yu, Yao, Jianhua, Shen, Xiaoyu, Zhang, Qiang
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
par: Yao, Lin
Publié: (2026)

Empirical Analysis of Decoding Biases in Masked Diffusion Models
par: Huang, Pengcheng, et autres
Publié: (2025)

Intermittent Semi-Working Mask: A New Masking Paradigm for LLMs
par: Hu, HaoYuan, et autres
Publié: (2024)

Exploration of Masked and Causal Language Modelling for Text Generation
par: Micheletti, Nicolo, et autres
Publié: (2024)

Entropy-aware Masking for Masked Language Modeling
par: Srinivasagan, Gokul, et autres
Publié: (2026)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
par: Liang, Wen, et autres
Publié: (2024)

Chinese ModernBERT with Whole-Word Masking
par: Zhao, Zeyu, et autres
Publié: (2025)

Token Masking Improves Transformer-Based Text Classification
par: Xu, Xianglong, et autres
Publié: (2025)

MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
par: Deng, Jingyuan, et autres
Publié: (2025)

Variational Masked Diffusion Models
par: Zhang, Yichi, et autres
Publié: (2025)

Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
par: Chen, Ziyu, et autres
Publié: (2025)

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token
par: Lin, Ailiang, et autres
Publié: (2025)

Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
par: Zheng, Kaiwen, et autres
Publié: (2024)

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
par: Zheng, Bo, et autres
Publié: (2026)

Mask-guided BERT for Few Shot Text Classification
par: Liao, Wenxiong, et autres
Publié: (2023)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
par: Qin, Jiayu, et autres
Publié: (2025)

Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step
par: Yang, Jingyi, et autres
Publié: (2025)

SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
par: Wang, Chenyu, et autres
Publié: (2025)

Inconsistencies in Masked Language Models
par: Young, Tom, et autres
Publié: (2022)

RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts
par: Yang, Jing, et autres
Publié: (2024)

Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models
par: Zhong, Linhao, et autres
Publié: (2026)

Masked Diffusion Models as Energy Minimization
par: Chen, Sitong, et autres
Publié: (2025)

How Do Humans Write Code? Large Models Do It the Same Way Too
par: Li, Long, et autres
Publié: (2024)

Memorization in Attention-only Transformers
par: Dana, Léo, et autres
Publié: (2024)

Scaling up Masked Diffusion Models on Text
par: Nie, Shen, et autres
Publié: (2024)

Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
par: Chen, Qian, et autres
Publié: (2023)

Dynamic Masking Rate Schedules for MLM Pretraining
par: Ankner, Zachary, et autres
Publié: (2023)

Infinite Mask Diffusion for Few-Step Distillation
par: Yoo, Jaehoon, et autres
Publié: (2026)

Defensive Dual Masking for Robust Adversarial Defense
par: Yang, Wangli, et autres
Publié: (2024)

Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling
par: Kesgin, Himmet Toprak, et autres
Publié: (2024)

Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models
par: Wang, Shuxun, et autres
Publié: (2025)

MiSS: Revisiting the Trade-off in LoRA with an Efficient Shard-Sharing Structure
par: Kang, Jiale, et autres
Publié: (2024)

Alignment-Enhanced Decoding:Defending via Token-Level Adaptive Refining of Probability Distributions
par: Liu, Quan, et autres
Publié: (2024)

Time-Aware Feature Selection: Adaptive Temporal Masking for Stable Sparse Autoencoder Training
par: Li, T. Ed, et autres
Publié: (2025)

ShardMemo: Masked MoE Routing for Sharded Agentic LLM Memory
par: Zhao, Yang, et autres
Publié: (2026)

Probing the Difficulty Perception Mechanism of Large Language Models
par: Lee, Sunbowen, et autres
Publié: (2025)

Reflection-Window Decoding: Text Generation with Selective Refinement
par: Tang, Zeyu, et autres
Publié: (2025)

Debiasing LLMs by Masking Unfairness-Driving Attention Heads
par: Han, Tingxu, et autres
Publié: (2025)

EMBRE: Entity-aware Masking for Biomedical Relation Extraction
par: Li, Mingjie, et autres
Publié: (2024)

DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
par: Yu, Longxuan, et autres
Publié: (2026)