Continuous First, Discrete Later: VQ-VAEs Without Dimensional Collapse

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Xinyu, Karagodin, Nikita, Hassani, Hamed, Hersek, Sinan, Liang, Paul Pu, Polyanskiy, Yury
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Clustering in Causal Attention Masking
by: Karagodin, Nikita, et al.
Published: (2024)

Normalization in Attention Dynamics
by: Karagodin, Nikita, et al.
Published: (2025)

Information-theoretic Generalization Analysis for VQ-VAEs: A Role of Latent Variables
by: Futami, Futoshi, et al.
Published: (2025)

SoundSculpt: Direction and Semantics Driven Ambisonic Target Sound Extraction
by: Chen, Tuochao, et al.
Published: (2025)

The Sample Complexity of Approximate Rejection Sampling with Applications to Smoothed Online Learning
by: Block, Adam, et al.
Published: (2023)

High-Rate Quantized Matrix Multiplication II
by: Ordentlich, Or, et al.
Published: (2026)

Is Dimensionality a Barrier for Retrieval Models?
by: Bangachev, Kiril, et al.
Published: (2026)

Nonparametric MLE for Gaussian Location Mixtures: Certified Computation and Generic Behavior
by: Polyanskiy, Yury, et al.
Published: (2025)

Mutual Information Collapse Explains Disentanglement Failure in $β$-VAEs
by: Vu, Minh, et al.
Published: (2026)

Optimal Quantization for Matrix Multiplication
by: Ordentlich, Or, et al.
Published: (2024)

A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
by: Zhang, Zegu, et al.
Published: (2026)

Price of universality in vector quantization is at most 0.11 bit
by: Harbuzova, Alina, et al.
Published: (2026)

Solving Empirical Bayes via Transformers
by: Teh, Anzo, et al.
Published: (2025)

Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent
by: Moniri, Behrad, et al.
Published: (2026)

On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective
by: Moniri, Behrad, et al.
Published: (2025)

Improved Variational Inference in Discrete VAEs using Error Correcting Codes
by: Martínez-García, María, et al.
Published: (2024)

Representation Alignment Rests on Linear Structure
by: Bangachev, Kiril, et al.
Published: (2026)

On the Minimax Regret of Sequential Probability Assignment via Square-Root Entropy
by: Jia, Zeyu, et al.
Published: (2025)

A Gapped Scale-Sensitive Dimension and Lower Bounds for Offset Rademacher Complexity
by: Jia, Zeyu, et al.
Published: (2025)

SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer
by: Chen, Hao, et al.
Published: (2024)

Arch-VQ: Discrete Architecture Representation Learning with Autoregressive Priors
by: Poddenige, Deshani Geethika, et al.
Published: (2025)

Universal priors: solving empirical Bayes via Bayesian inference and pretraining
by: Cannella, Nick, et al.
Published: (2026)

Atoms as Language: VQ-Atom: Semantic Discretization for Molecular Representation Learning
by: Kimura, Takayuki
Published: (2026)

Toward Architecture-Agnostic Local Control of Posterior Collapse in VAEs
by: Song, Hyunsoo, et al.
Published: (2025)

YuriiFormer: A Suite of Nesterov-Accelerated Transformers
by: Zimin, Aleksandr, et al.
Published: (2026)

The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression
by: Hassani, Hamed, et al.
Published: (2022)

Asymptotics of Linear Regression with Linearly Dependent Data
by: Moniri, Behrad, et al.
Published: (2024)

High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curve
by: Ichikawa, Yuma, et al.
Published: (2023)

WaterSIC: information-theoretically (near) optimal linear layer quantization
by: Lifar, Egor, et al.
Published: (2026)

Improving the Generation of VAEs with High Dimensional Latent Spaces by the use of Hyperspherical Coordinates
by: Ascarate, Alejandro, et al.
Published: (2025)

Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs
by: Naiman, Ilan, et al.
Published: (2023)

Deep Incomplete Multi-view Learning via Cyclic Permutation of VAEs
by: Gao, Xin, et al.
Published: (2025)

Synchronization of mean-field models on the circle
by: Polyanskiy, Yury, et al.
Published: (2025)

Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization
by: Joshi, Sunay, et al.
Published: (2025)

Signal-Plus-Noise Decomposition of Nonlinear Spiked Random Matrix Models
by: Moniri, Behrad, et al.
Published: (2024)

Watermark Smoothing Attacks against Language Models
by: Chang, Hongyan, et al.
Published: (2024)

The emergence of clusters in self-attention dynamics
by: Geshkovski, Borjan, et al.
Published: (2023)

Global Minimizers of Sigmoid Contrastive Loss
by: Bangachev, Kiril, et al.
Published: (2025)

NestQuant: Nested Lattice Quantization for Matrix Products and LLMs
by: Savkin, Semyon, et al.
Published: (2025)

Prior Learning in Introspective VAEs
by: Athanasiadis, Ioannis, et al.
Published: (2024)