Image de couverture de livre

Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Wenhao, Zou, Qiran, Shah, Rushi, Wu, Yudi, Lin, Zhouhan, Liu, Dianbo
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Representation Collapsing Problems in Vector Quantization
par: Zhao, Wenhao, et autres
Publié: (2024)

Mitigating Premature Discretization with Progressive Quantization for Robust Vector Tokenization
par: Zhao, Wenhao, et autres
Publié: (2026)

Deconstructing Generative Diversity: An Information Bottleneck Analysis of Discrete Latent Generative Models
par: Wu, Yudi, et autres
Publié: (2025)

Improving Discrete Optimisation Via Decoupled Straight-Through Estimator
par: Shah, Rushi, et autres
Publié: (2024)

Auto-Discovery-Bench: Diagnosing Structured State Tracking in Oracle-Guided Discovery
par: Chen, Tingting, et autres
Publié: (2025)

Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior
par: Yan, Mingyuan, et autres
Publié: (2024)

VQSynery: Robust Drug Synergy Prediction With Vector Quantization Mechanism
par: Wu, Jiawei, et autres
Publié: (2024)

Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm
par: Yang, Kaisen, et autres
Publié: (2025)

Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
par: Sinha, Abhijeet, et autres
Publié: (2026)

Performance Asymmetry in Model-Based Reinforcement Learning
par: Lim, Jing Yu, et autres
Publié: (2025)

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
par: Zou, Qiran, et autres
Publié: (2026)

CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMs
par: Park, Gunho, et autres
Publié: (2025)

SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation
par: Vetcha, Nitin, et autres
Publié: (2026)

QSpec: Speculative Decoding with Complementary Quantization Schemes
par: Zhao, Juntao, et autres
Publié: (2024)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
par: Shen, Junzhe, et autres
Publié: (2026)

ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
par: He, Yefei, et autres
Publié: (2024)

On the Structural Limitations of Weight-Based Neural Adaptation and the Role of Reversible Behavioral Learning
par: Konduru, Pardhu Sri Rushi Varma
Publié: (2026)

Rep2Text: Decoding Full Text from a Single LLM Token Representation
par: Zhao, Haiyan, et autres
Publié: (2025)

Evolution Guided Generative Flow Networks
par: Ikram, Zarif, et autres
Publié: (2024)

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
par: Tseng, Albert, et autres
Publié: (2024)

TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
par: Zhang, Yuxiang, et autres
Publié: (2025)

Collaborative Deterministic-Probabilistic Forecasting for Diverse Spatiotemporal Systems
par: Sheng, Zhi, et autres
Publié: (2025)

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
par: Fu, Yuqian, et autres
Publié: (2026)

Preserve-Then-Quantize: Balancing Rank Budgets for Quantization Error Reconstruction in LLMs
par: Cho, Yoonjun, et autres
Publié: (2026)

Enhancing Adversarial Transferability through Block Stretch and Shrink
par: Liu, Quan, et autres
Publié: (2025)

Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design
par: Zhang, Yudi, et autres
Publié: (2025)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
par: Zhang, Yaxiang, et autres
Publié: (2026)

FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking
par: Rao, Xuan, et autres
Publié: (2025)

Learning Graph Quantized Tokenizers
par: Wang, Limei, et autres
Publié: (2024)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
par: Lu, Liming, et autres
Publié: (2026)

STORI: A Benchmark and Taxonomy for Stochastic Environments
par: Barsainyan, Aryan Amit, et autres
Publié: (2025)

Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities
par: Xie, Ziwen, et autres
Publié: (2026)

Data Classification with Dynamically Growing and Shrinking Neural Networks
par: Świderski, Szymon, et autres
Publié: (2025)

FoldToken: Learning Protein Language via Vector Quantization and Beyond
par: Gao, Zhangyang, et autres
Publié: (2024)

Critical Data Size of Language Models from a Grokking Perspective
par: Zhu, Xuekai, et autres
Publié: (2024)

Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective
par: Zhang, Yi-Ge, et autres
Publié: (2025)

Taming the Entropy Cliff: Variable Codebook Size Quantization for Autoregressive Visual Generation
par: Zheng, Bowen, et autres
Publié: (2026)

Patch-aware Vector Quantized Codebook Learning for Unsupervised Visual Defect Detection
par: Cheng, Qisen, et autres
Publié: (2025)

Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization
par: Della Libera, Luca, et autres
Publié: (2026)

RSQ: Learning from Important Tokens Leads to Better Quantized LLMs
par: Sung, Yi-Lin, et autres
Publié: (2025)