Cover Image

The Inductive Bottleneck: Data-Driven Emergence of Representational Sparsity in Vision Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Awadhiya, Kanishk
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Aiding Medical Diagnosis through Image Synthesis and Classification
by: Choudhary, Kanishk
Published: (2025)

Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization
by: Awadhiya, Kanishk
Published: (2026)

Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformers
by: Hong, Jung-Ho, et al.
Published: (2025)

Learning Unsupervised Gaze Representation via Eye Mask Driven Information Bottleneck
by: Jiang, Yangzhou, et al.
Published: (2024)

SINR: Sparsity Driven Compressed Implicit Neural Representations
by: Jayasundara, Dhananjaya, et al.
Published: (2025)

Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
by: Ye, Hancheng, et al.
Published: (2024)

TAB: Transformer Attention Bottlenecks enable User Intervention and Debugging in Vision-Language Models
by: Rahmanzadehgervi, Pooyan, et al.
Published: (2024)

Learnable Sparsity for Vision Generative Models
by: Zhang, Yang, et al.
Published: (2024)

On Inductive Biases That Enable Generalization of Diffusion Transformers
by: An, Jie, et al.
Published: (2024)

LIB-KD: Teaching Inductive Bias for Efficient Vision Transformer Distillation and Compression
by: Habib, Gousia, et al.
Published: (2023)

MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective
by: Wang, Weitian, et al.
Published: (2025)

Representation Forcing for Bottleneck-Free Unified Multimodal Models
by: Wang, Yuqing, et al.
Published: (2026)

SPFormer: Enhancing Vision Transformer with Superpixel Representation
by: Mei, Jieru, et al.
Published: (2024)

ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration
by: Huang, Ning-Chi, et al.
Published: (2024)

LAPA: Log-Domain Prediction-Driven Dynamic Sparsity Accelerator for Transformer Model
by: Wang, Huizheng, et al.
Published: (2025)

Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces
by: Bond, Andrew, et al.
Published: (2026)

VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning
by: Xu, Hengbo, et al.
Published: (2026)

Exploring Compositionality in Vision Transformers using Wavelet Representations
by: Purushottamdas, Akshad Shyam, et al.
Published: (2025)

Analyzing Local Representations of Self-supervised Vision Transformers
by: Vanyan, Ani, et al.
Published: (2023)

Distilling Vision Transformers for Distortion-Robust Representation Learning
by: Alexis, Konstantinos, et al.
Published: (2026)

Improving Concept Alignment in Vision-Language Concept Bottleneck Models
by: Selvaraj, Nithish Muthuchamy, et al.
Published: (2024)

Diagnosing Bottlenecks in Data Visualization Understanding by Vision-Language Models
by: Tartaglini, Alexa R., et al.
Published: (2025)

Information-Bottleneck Driven Binary Neural Network for Change Detection
by: Yin, Kaijie, et al.
Published: (2025)

DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
by: Zhu, Haowei, et al.
Published: (2026)

From Colors to Classes: Emergence of Concepts in Vision Transformers
by: Dorszewski, Teresa, et al.
Published: (2025)

A Hidden Semantic Bottleneck in Conditional Embeddings of Diffusion Transformers
by: Pham, Trung X., et al.
Published: (2026)

Calibration Attention: Learning Reliability-Aware Representations for Vision Transformers
by: Liang, Wenhao, et al.
Published: (2025)

Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathology
by: Liu, Pei, et al.
Published: (2024)

HeBA: Heterogeneous Bottleneck Adapters for Robust Vision-Language Models
by: Islam, Md Jahidul
Published: (2026)

Concepts from Representations: Post-hoc Concept Bottleneck Models via Sparse Decomposition of Visual Representations
by: Gong, Shizhan, et al.
Published: (2026)

High-Fidelity Differential-information Driven Binary Vision Transformer
by: Gao, Tian, et al.
Published: (2025)

Data-Efficient Challenges in Visual Inductive Priors: A Retrospective
by: Bruintjes, Robert-Jan, et al.
Published: (2025)

IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning
by: Xiang, Canqun, et al.
Published: (2026)

DCBM: Data-Efficient Visual Concept Bottleneck Models
by: Prasse, Katharina, et al.
Published: (2024)

Disentangled Representation Learning with Transmitted Information Bottleneck
by: Dang, Zhuohang, et al.
Published: (2023)

Unsqueeze [CLS] Bottleneck to Learn Rich Representations
by: Su, Qing, et al.
Published: (2024)

Convolutional Initialization for Data-Efficient Vision Transformers
by: Zheng, Jianqiao, et al.
Published: (2024)

V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
by: He, Hangzhou, et al.
Published: (2025)

Discovering Failure Modes in Vision-Language Models using RL
by: Jain, Kanishk, et al.
Published: (2026)

PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment
by: Liu, Zhendong, et al.
Published: (2024)