Cover Image

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Bo, Chen, Yudong, Xiong, Zihua, Fang, Shuai, He, Peidong, Yang, Yang, Guo, Sheng
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification
by: Dissanayake, Pasan, et al.
Published: (2025)

Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs
by: Edman, Lukas, et al.
Published: (2025)

UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science
by: Yang, Yazheng, et al.
Published: (2023)

LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking
by: Elgaar, Mohamed, et al.
Published: (2024)

Dynamic Masking Rate Schedules for MLM Pretraining
by: Ankner, Zachary, et al.
Published: (2023)

CausalDiffTab: Mixed-Type Causal-Aware Diffusion for Tabular Data Generation
by: Zhang, Jia-Chen, et al.
Published: (2025)

Defensive Dual Masking for Robust Adversarial Defense
by: Yang, Wangli, et al.
Published: (2024)

Should We Still Pretrain Encoders with Masked Language Modeling?
by: Gisserot-Boukhlef, Hippolyte, et al.
Published: (2025)

Inconsistencies in Masked Language Models
by: Young, Tom, et al.
Published: (2022)

Mask-guided BERT for Few Shot Text Classification
by: Liao, Wenxiong, et al.
Published: (2023)

Scaling Beyond Masked Diffusion Language Models
by: Sahoo, Subham Sekhar, et al.
Published: (2026)

Infinite Mask Diffusion for Few-Step Distillation
by: Yoo, Jaehoon, et al.
Published: (2026)

MIDI-to-Tab: Guitar Tablature Inference via Masked Language Modeling
by: Edwards, Drew, et al.
Published: (2024)

Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis
by: An, Seunghwan, et al.
Published: (2024)

MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion
by: Fan, Cunhang, et al.
Published: (2024)

Masked Language Models are Good Heterogeneous Graph Generalizers
by: Yang, Jinyu, et al.
Published: (2025)

StableMask: Refining Causal Masking in Decoder-only Transformer
by: Yin, Qingyu, et al.
Published: (2024)

MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
by: Deng, Jingyuan, et al.
Published: (2025)

Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation
by: Ma, Ziyang, et al.
Published: (2025)

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
by: Liu, Hongyi, et al.
Published: (2025)

Intermittent Semi-Working Mask: A New Masking Paradigm for LLMs
by: Hu, HaoYuan, et al.
Published: (2024)

Disentangling Dual-Encoder Masked Autoencoder for Respiratory Sound Classification
by: Wei, Peidong, et al.
Published: (2025)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

TabVer: Tabular Fact Verification with Natural Logic
by: Aly, Rami, et al.
Published: (2024)

Token Masking Improves Transformer-Based Text Classification
by: Xu, Xianglong, et al.
Published: (2025)

Entropy-aware Masking for Masked Language Modeling
by: Srinivasagan, Gokul, et al.
Published: (2026)

TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields
by: Arazi, Alan, et al.
Published: (2025)

Robust Evaluation Measures for Evaluating Social Biases in Masked Language Models
by: Liu, Yang
Published: (2024)

Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining
by: Zhang, Qi, et al.
Published: (2024)

Improving Sampling for Masked Diffusion Models via Information Gain
by: Yang, Kaisen, et al.
Published: (2026)

Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
by: Zheng, Kaiwen, et al.
Published: (2024)

TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems
by: Liu, Si-Yang, et al.
Published: (2025)

$Δ$-AttnMask: Attention-Guided Masked Hidden States for Efficient Data Selection and Augmentation
by: Hu, Jucheng, et al.
Published: (2025)

Are Emotion and Rhetoric Neurons in LLM? Neuron Recognition and Adaptive Masking for Emotion-Rhetoric Prediction Steering
by: Zheng, Li, et al.
Published: (2026)

Good Arguments Against the People Pleasers: How Reasoning Mitigates (Yet Masks) LLM Sycophancy
by: Feng, Zhaoxin, et al.
Published: (2026)

Scaling up Masked Diffusion Models on Text
by: Nie, Shen, et al.
Published: (2024)

DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
by: Yu, Longxuan, et al.
Published: (2026)

MaskLID: Code-Switching Language Identification through Iterative Masking
by: Kargaran, Amir Hossein, et al.
Published: (2024)

TabReX : Tabular Referenceless eXplainable Evaluation
by: Anvekar, Tejas, et al.
Published: (2025)