Cover Image

Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bouadi, Mohamed, Seth, Pratinav, Tanna, Aditya, Sankarapu, Vinay Kumar
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning
by: Bouadi, Mohamed, et al.
Published: (2025)

Distilling Tabular Foundation Models for Structured Health Data
by: Tanna, Aditya, et al.
Published: (2026)

TabTune: A Unified Library for Inference and Fine-Tuning Tabular Foundation Models
by: Tanna, Aditya, et al.
Published: (2025)

Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap
by: Tanna, Aditya, et al.
Published: (2026)

Data Presentation Over Architecture: Resampling Strategies for Credit Risk Prediction with Tabular Foundation Models
by: Tanna, Aditya, et al.
Published: (2026)

Exploring Fine-Tuning for Tabular Foundation Models
by: Tanna, Aditya, et al.
Published: (2026)

Pocket Foundation Models: Distilling TFMs into CPU-Ready Gradient-Boosted Trees
by: Tanna, Aditya, et al.
Published: (2026)

Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Quality
by: Bouadi, Mohamed, et al.
Published: (2026)

Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands
by: Seth, Pratinav, et al.
Published: (2026)

Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliance
by: Seth, Pratinav, et al.
Published: (2025)

Interpretability as Alignment: Making Internal Understanding a Design Principle
by: Sengupta, Aadit, et al.
Published: (2025)

DLBacktrace: A Model Agnostic Explainability for any Deep Learning Models
by: Sankarapu, Vinay Kumar, et al.
Published: (2024)

xai_evals : A Framework for Evaluating Post-Hoc Local Explanation Methods
by: Seth, Pratinav, et al.
Published: (2025)

Interpretability-Aware Pruning for Efficient Medical Image Analysis
by: Malik, Nikita, et al.
Published: (2025)

Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
by: Sadhu, Saisab, et al.
Published: (2026)

Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoning
by: Yuan, Rui, et al.
Published: (2026)

Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization
by: Khandoga, Mykola, et al.
Published: (2026)

$C$-$ΔΘ$: Circuit-Restricted Weight Arithmetic for Selective Refusal
by: Kasliwal, Aditya, et al.
Published: (2026)

Leveraging Knowlegde Graphs for Interpretable Feature Generation
by: Bouadi, Mohamed, et al.
Published: (2024)

vAttention: Verified Sparse Attention
by: Desai, Aditya, et al.
Published: (2025)

Noise Immunity in In-Context Tabular Learning: An Empirical Robustness Analysis of TabPFN's Attention Mechanisms
by: Hu, James, et al.
Published: (2026)

MultiMax: Sparse and Multi-Modal Attention Learning
by: Zhou, Yuxuan, et al.
Published: (2024)

Multi-branch of Attention Yields Accurate Results for Tabular Data
by: Li, Xuechen, et al.
Published: (2025)

SparseBalance: Load-Balanced Long Context Training with Dynamic Sparse Attention
by: Xu, Hongtao, et al.
Published: (2026)

Scaling Context Requires Rethinking Attention
by: Gelada, Carles, et al.
Published: (2025)

Multi-Layer Attention-Based Explainability via Transformers for Tabular Data
by: Gavito, Andrea Treviño, et al.
Published: (2023)

Towards Understanding Layer Contributions in Tabular In-Context Learning Models
by: Balef, Amir Rezaei, et al.
Published: (2025)

Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees
by: Nobaub, Vashista
Published: (2026)

TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning
by: Zou, Jiaru, et al.
Published: (2025)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
by: Zhou, Ruijie, et al.
Published: (2026)

Improving LLM Group Fairness on Tabular Data via In-Context Learning
by: Cherepanova, Valeriia, et al.
Published: (2024)

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
by: Ni, Wentao, et al.
Published: (2026)

VSPrefill: Vertical-Slash Sparse Attention with Lightweight Indexing for Long-Context Prefilling
by: Guanzhong, Chen
Published: (2026)

TabICL: A Tabular Foundation Model for In-Context Learning on Large Data
by: Qu, Jingang, et al.
Published: (2025)

Scaling Graph Transformers: A Comparative Study of Sparse and Dense Attention
by: Dimitrov, Leon
Published: (2025)

LATTLE: LLM Attention Transplant for Transfer Learning of Tabular Data Across Disparate Domains
by: Kowsar, Ibna, et al.
Published: (2025)

Mitigating Label Shift in Tabular In-Context Learning via Test-Time Posterior Adjustment
by: Lee, Seunghan
Published: (2026)

BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model
by: Xu, Chenwei, et al.
Published: (2024)

Adaptive Weighted Loss for Sequential Recommendations on Sparse Domains
by: Mittal, Akshay, et al.
Published: (2025)

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
by: Liu, Renpu, et al.
Published: (2024)