Image de couverture de livre

Zonkey: A Hierarchical Diffusion Language Model with Differentiable Tokenization and Probabilistic Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Rozental, Alon
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
par: Levy, Mosh, et autres
Publié: (2024)

Probabilistic Token Alignment for Large Language Model Fusion
par: Zeng, Runjia, et autres
Publié: (2025)

Certain but not Probable? Differentiating Certainty from Probability in LLM Token Outputs for Probabilistic Scenarios
par: Toney-Wails, Autumn, et autres
Publié: (2025)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
par: Huang, Yuxiang, et autres
Publié: (2026)

SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
par: Wang, Zekun, et autres
Publié: (2023)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
par: Zhang, Zihou, et autres
Publié: (2026)

Probabilistic Lexical Manifold Construction in Large Language Models via Hierarchical Vector Field Interpolation
par: Pendleton, Clive, et autres
Publié: (2025)

Attention Sinks in Diffusion Language Models
par: Rulli, Maximo Eduardo, et autres
Publié: (2025)

DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
par: Zheng, Caleb, et autres
Publié: (2026)

SparseD: Sparse Attention for Diffusion Language Models
par: Wang, Zeqing, et autres
Publié: (2025)

Multi-Token Attention
par: Golovneva, Olga, et autres
Publié: (2025)

HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models
par: Guo, Yansong, et autres
Publié: (2026)

Impact of Preference Noise on the Alignment Performance of Generative Language Models
par: Gao, Yang, et autres
Publié: (2024)

Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models
par: Yao, Lin
Publié: (2026)

H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages
par: Zakershahrak, Mehrdad, et autres
Publié: (2025)

Attention-Based Sampler for Diffusion Language Models
par: Zhou, Yuyan, et autres
Publié: (2026)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
par: Wu, Zihao, et autres
Publié: (2026)

Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
par: Yao, Lin
Publié: (2026)

Adaptive Targeted Dynamic Chunking for Tokenization-Free Hierarchical Model
par: Dang, Thang, et autres
Publié: (2026)

Multimodal Latent Language Modeling with Next-Token Diffusion
par: Sun, Yutao, et autres
Publié: (2024)

Differentially Private Next-Token Prediction of Large Language Models
par: Flemings, James, et autres
Publié: (2024)

MBTSAD: Mitigating Backdoors in Language Models Based on Token Splitting and Attention Distillation
par: Ding, Yidong, et autres
Publié: (2025)

CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling
par: Singaravadivelan, Karthik, et autres
Publié: (2026)

Problematic Tokens: Tokenizer Bias in Large Language Models
par: Yang, Jin, et autres
Publié: (2024)

EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
par: Cheong, Minsoo, et autres
Publié: (2026)

Token Perturbation Guidance for Diffusion Models
par: Rajabi, Javad, et autres
Publié: (2025)

PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language Model
par: Arif, Kazi Hasan Ibn, et autres
Publié: (2025)

Just on Time: Token-Level Early Stopping for Diffusion Language Models
par: Kohut, Zahar, et autres
Publié: (2026)

Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models
par: Zhong, Linhao, et autres
Publié: (2026)

Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
par: Yang, Jinbiao
Publié: (2024)

Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding
par: Sun, Bowen, et autres
Publié: (2025)

HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignment
par: Chen, Yongqiang, et autres
Publié: (2024)

The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
par: Sun, Bohang, et autres
Publié: (2026)

Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning
par: Ye, Ziang, et autres
Publié: (2024)

A Hierarchical Probabilistic Framework for Incremental Knowledge Tracing in Classroom Settings
par: Gao, Xinyi, et autres
Publié: (2025)

Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns
par: DuSell, Brian, et autres
Publié: (2023)

Multiscale Aggregated Hierarchical Attention (MAHA): A Game Theoretic and Optimization Driven Approach to Efficient Contextual Modeling in Large Language Models
par: Erden, Caner
Publié: (2025)

Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobs
par: Wagner, Eitan, et autres
Publié: (2025)

Token-weighted Direct Preference Optimization with Attention
par: Huang, Chengyu, et autres
Publié: (2026)

DP-Fusion: Token-Level Differentially Private Inference for Large Language Models
par: Thareja, Rushil, et autres
Publié: (2025)