Cover Image

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Yaodong, Buchanan, Sam, Pai, Druv, Chu, Tianzhe, Wu, Ziyang, Tong, Shengbang, Bai, Hao, Zhai, Yuexiang, Haeffele, Benjamin D., Ma, Yi
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Masked Completion via Structured Diffusion with White-Box Transformers
by: Pai, Druv, et al.
Published: (2024)

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models
by: Chu, Tianzhe, et al.
Published: (2023)

Scaling White-Box Transformers for Vision
by: Yang, Jinrui, et al.
Published: (2024)

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction
by: Wu, Ziyang, et al.
Published: (2024)

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs
by: Yeh, Chun-Hsiao, et al.
Published: (2025)

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
by: Chu, Tianzhe, et al.
Published: (2025)

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
by: Tong, Shengbang, et al.
Published: (2024)

Simplifying DINO via Coding Rate Regularization
by: Wu, Ziyang, et al.
Published: (2025)

A Global Geometric Analysis of Maximal Coding Rate Reduction
by: Wang, Peng, et al.
Published: (2024)

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
by: Zhai, Yuexiang, et al.
Published: (2024)

On the Edge of Memorization in Diffusion Models
by: Buchanan, Sam, et al.
Published: (2025)

Diffusion Transformers with Representation Autoencoders
by: Zheng, Boyang, et al.
Published: (2025)

Sparse Transformer for Ultra-sparse Sampled Video Compressive Sensing
by: Cao, Miao, et al.
Published: (2025)

Attention-Only Transformers via Unrolled Subspace Denoising
by: Wang, Peng, et al.
Published: (2025)

PRISM: Deriving a White-Box Transformer as a Signal-Noise Decomposition Operator via Maximum Coding Rate Reduction
by: Huang, Dongchen
Published: (2026)

Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning
by: Mo, Shentong, et al.
Published: (2024)

Mass-Producing Failures of Multimodal Systems with Language Models
by: Tong, Shengbang, et al.
Published: (2023)

Independent and Decentralized Learning in Markov Potential Games
by: Maheshwari, Chinmay, et al.
Published: (2022)

Congestion Pricing for Efficiency and Equity: Theory and Applications to the San Francisco Bay Area
by: Maheshwari, Chinmay, et al.
Published: (2024)

Learning Hierarchical Sparse Transform Coding for 3DGS Compression
by: Xu, Hao, et al.
Published: (2025)

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
by: Fang, Irving, et al.
Published: (2025)

Motif Channel Opened in a White-Box: Stereo Matching via Motif Correlation Graph
by: Chen, Ziyang, et al.
Published: (2024)

White-Box Sensitivity Auditing with Steering Vectors
by: Cyberey, Hannah, et al.
Published: (2026)

From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images
by: Chen, Yiming, et al.
Published: (2025)

Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction
by: Wang, Hong, et al.
Published: (2022)

Asymmetric Idiosyncrasies in Multimodal Models
by: Tao, Muzi, et al.
Published: (2026)

VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images
by: Zhou, Guanyu, et al.
Published: (2026)

Adaptive Stain Normalization for Cross-Domain Medical Histology
by: Xu, Tianyue, et al.
Published: (2025)

DeepTutor: Towards Agentic Personalized Tutoring
by: Zhao, Bingxi, et al.
Published: (2026)

Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models
by: Nagle, Alliot, et al.
Published: (2024)

Temporal Rate Reduction Clustering for Human Motion Segmentation
by: Meng, Xianghan, et al.
Published: (2025)

Learning Dual Transformers for All-In-One Image Restoration from a Frequency Perspective
by: Chu, Jie, et al.
Published: (2024)

Improving Neuron-level Interpretability with White-box Language Models
by: Bai, Hao, et al.
Published: (2024)

ICPC: In-context Prompt Compression with Faster Inference
by: Yu, Ziyang, et al.
Published: (2025)

Compression of Voxelized Vector Field Data by Boxes is Hard
by: Zhang, Simon
Published: (2025)

1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models
by: Zong, Zeliang, et al.
Published: (2025)

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
by: Tong, Shengbang, et al.
Published: (2026)

Towards White Box Deep Learning
by: Satkiewicz, Maciej
Published: (2024)

What's in a Prior? Learned Proximal Networks for Inverse Problems
by: Fang, Zhenghan, et al.
Published: (2023)

Black-Box Crypto is Useless for Pseudorandom Codes
by: Garg, Sanjam, et al.
Published: (2025)