Cover Image

TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dissanayake, Pasan, Dutta, Sanghamitra
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
by: Hamman, Faisal, et al.
Published: (2025)

Quantifying Prediction Consistency Under Fine-Tuning Multiplicity in Tabular LLMs
by: Hamman, Faisal, et al.
Published: (2024)

Learning Invariant Graph Representations Through Redundant Information
by: Halder, Barproda, et al.
Published: (2025)

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
by: Zheng, Bo, et al.
Published: (2026)

Model Reconstruction Using Counterfactual Explanations: A Perspective From Polytope Theory
by: Dissanayake, Pasan, et al.
Published: (2024)

Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition
by: Halder, Barproda, et al.
Published: (2024)

Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering
by: Gupta, Akash, et al.
Published: (2025)

Quantifying Knowledge Distillation Using Partial Information Decomposition
by: Dissanayake, Pasan, et al.
Published: (2024)

Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
by: Shen, William, et al.
Published: (2023)

Manual Verbalizer Enrichment for Few-Shot Text Classification
by: Nguyen, Quang Anh, et al.
Published: (2024)

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation
by: Vöge, Lukas, et al.
Published: (2024)

Demystifying the Accuracy-Interpretability Trade-Off: A Case Study of Inferring Ratings from Reviews
by: Atrey, Pranjal, et al.
Published: (2025)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
by: Monsefi, Amin Karimi, et al.
Published: (2026)

Counterfactual Explanations for Model Ensembles Using Entropic Risk Measures
by: Noorani, Erfaun, et al.
Published: (2025)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation
by: Phan, Phuc, et al.
Published: (2024)

Merge-of-Thought Distillation
by: Shen, Zhanming, et al.
Published: (2025)

Distillation Scaling Laws
by: Busbridge, Dan, et al.
Published: (2025)

Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression
by: Zhou, Longsheng, et al.
Published: (2026)

TabDLM: Free-Form Tabular Data Generation via Joint Numerical-Language Diffusion
by: Cai, Donghong, et al.
Published: (2026)

UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science
by: Yang, Yazheng, et al.
Published: (2023)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation
by: Singh, Aasheesh, et al.
Published: (2025)

VisTabNet: Adapting Vision Transformers for Tabular Data
by: Wydmański, Witold, et al.
Published: (2024)

Few-Shot Recalibration of Language Models
by: Li, Xiang Lisa, et al.
Published: (2024)

Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers
by: Grigore, Diana-Nicoleta, et al.
Published: (2024)

FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answering
by: Abaskohi, Amirhossein, et al.
Published: (2024)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

Automatic Prompt Optimization with Prompt Distillation
by: Dyagin, Ernest A., et al.
Published: (2025)

Efficiently Distilling LLMs for Edge Applications
by: Kundu, Achintya, et al.
Published: (2024)

Self-Distilled Agentic Reinforcement Learning
by: Lu, Zhengxi, et al.
Published: (2026)

Ensemble Distillation for Unsupervised Constituency Parsing
by: Shayegh, Behzad, et al.
Published: (2023)

KVSculpt: KV Cache Compression as Distillation
by: Jiang, Bo, et al.
Published: (2026)

Self-Distillation for Multi-Token Prediction
by: Zhao, Guoliang, et al.
Published: (2026)

M2Distill: Multi-Modal Distillation for Lifelong Imitation Learning
by: Roy, Kaushik, et al.
Published: (2024)

Language Model Representations for Efficient Few-Shot Tabular Classification
by: Kang, Inwon, et al.
Published: (2026)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
by: Chen, Jennifer, et al.
Published: (2025)

Meta-Semantics Augmented Few-Shot Relational Learning
by: Wu, Han, et al.
Published: (2025)

Emergent Communication Pretraining for Few-Shot Machine Translation
by: Li, Yaoyiran, et al.
Published: (2020)

TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins
by: Pereira, Shovon Niverd, et al.
Published: (2026)