Cover Image

VTrans: Accelerating Transformer Compression with Variational Information Bottleneck based Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dutta, Oshin, Gupta, Ritvik, Agarwal, Sumeet
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Differential Privacy for Transformer Embeddings of Text with Nonparametric Variational Information Bottleneck
by: Zein, Dina El, et al.
Published: (2026)

Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training
by: Kudo, Sota, et al.
Published: (2024)

Variational Geometric Information Bottleneck: Learning the Shape of Understanding
by: Katende, Ronald
Published: (2025)

Memory Faults in Activation-sparse Quantized Deep Neural Networks: Analysis and Mitigation using Sharpness-aware Training
by: Malhotra, Akul, et al.
Published: (2024)

Variance-Based Pruning for Accelerating and Compressing Trained Networks
by: Berisha, Uranik, et al.
Published: (2025)

Domain Generalization In Robust Invariant Representation
by: Gupta, Gauri, et al.
Published: (2023)

Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck
by: Massoli, Fabio Valerio, et al.
Published: (2026)

Adaptive Test-Time Intervention for Concept Bottleneck Models
by: Shen, Matthew, et al.
Published: (2025)

Practical GPU Choices for Earth Observation: ResNet-50 Training Throughput on Integrated, Laptop, and Cloud Accelerators
by: Chaturvedi, Ritvik
Published: (2025)

Blockchain-Enabled Variational Information Bottleneck for Data Extraction Based on Mutual Information in Internet of Vehicles
by: Zhang, Cui, et al.
Published: (2024)

Is the Information Bottleneck Robust Enough? Towards Label-Noise Resistant Information Bottleneck Learning
by: Huang, Yi, et al.
Published: (2025)

Interpretable Prototype-based Graph Information Bottleneck
by: Seo, Sangwoo, et al.
Published: (2023)

RespoDiff: Dual-Module Bottleneck Transformation for Responsible & Faithful T2I Generation
by: Sreelatha, Silpa Vadakkeeveetil, et al.
Published: (2025)

A Distance Metric Learning Model Based On Variational Information Bottleneck
by: Zhang, YaoDan, et al.
Published: (2024)

Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses
by: Abdelaleem, Eslam, et al.
Published: (2023)

First 100 days of pandemic; an interplay of pharmaceutical, behavioral and digital interventions -- A study using agent based modeling
by: Gupta, Gauri, et al.
Published: (2024)

Low Power Vision Transformer Accelerator with Hardware-Aware Pruning and Optimized Dataflow
by: Hsiung, Ching-Lin, et al.
Published: (2025)

Concepts' Information Bottleneck Models
by: Galliamov, Karim, et al.
Published: (2026)

Comparative Evaluation of Memory Technologies for Synaptic Crossbar Arrays- Part 2: Design Knobs and DNN Accuracy Trends
by: Victor, Jeffry, et al.
Published: (2024)

GeoIB: Geometry-Aware Information Bottleneck via Statistical-Manifold Compression
by: Wang, Weiqi, et al.
Published: (2026)

Self-Explainable Temporal Graph Networks based on Graph Information Bottleneck
by: Seo, Sangwoo, et al.
Published: (2024)

PACE: Prune-And-Compress Ensemble Models
by: Akkerman, Fabian, et al.
Published: (2026)

Debiasing Graph Representation Learning based on Information Bottleneck
by: Zhang, Ziyi, et al.
Published: (2024)

QAdaPrune: Adaptive Parameter Pruning For Training Variational Quantum Circuits
by: Kulshrestha, Ankit, et al.
Published: (2024)

VNDUQE: Information-Theoretic Novelty Detection using Deep Variational Information Bottleneck
by: Gondkar, Aryan, et al.
Published: (2026)

Discrete Curvature Graph Information Bottleneck
by: Fu, Xingcheng, et al.
Published: (2024)

Pruning for Improved ADC Efficiency in Crossbar-based Analog In-memory Accelerators
by: Ibrayev, Timur, et al.
Published: (2024)

Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization
by: Jiang, Hao, et al.
Published: (2026)

No Free Prune: Information-Theoretic Barriers to Pruning at Initialization
by: Kumar, Tanishq, et al.
Published: (2024)

Refining the Information Bottleneck via Adversarial Information Separation
by: Ning, Shuai, et al.
Published: (2026)

Towards Reasonable Concept Bottleneck Models
by: Kalampalikis, Nektarios, et al.
Published: (2025)

Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation
by: Chen, Xizi, et al.
Published: (2021)

Adaptive Pruning with Module Robustness Sensitivity: Balancing Compression and Robustness
by: Bai, Lincen, et al.
Published: (2024)

SHRP: Specialized Head Routing and Pruning for Efficient Encoder Compression
by: Su, Zeli, et al.
Published: (2025)

BiPETE: A Bi-Positional Embedding Transformer Encoder for Risk Assessment of Alcohol and Substance Use Disorder with Electronic Health Records
by: Lee, Daniel S., et al.
Published: (2025)

MI-to-Mid Distilled Compression (M2M-DC): An Hybrid-Information-Guided-Block Pruning with Progressive Inner Slicing Approach to Model Compression
by: Levine, Lionel, et al.
Published: (2025)

Dynamic Graph Information Bottleneck
by: Yuan, Haonan, et al.
Published: (2024)

Aligning Multimodal Representations through an Information Bottleneck
by: Almudévar, Antonio, et al.
Published: (2025)

Learning Optimal Multimodal Information Bottleneck Representations
by: Wu, Qilong, et al.
Published: (2025)

SAP: Syntactic Attention Pruning for Transformer-based Language Models
by: Lee, Tzu-Yun, et al.
Published: (2025)