Copertina

Improving Variable-Length Generation in Diffusion Language Models via Length Regularization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cheng, Zicong, Jia, Ruixuan, Li, Jia, Yang, Guo-Wei, Guo, Meng-Hao, Hu, Shi-Min
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DEER: Draft with Diffusion, Verify with Autoregressive Models
di: Cheng, Zicong, et al.
Pubblicazione: (2025)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
di: Leng, Jiaqi, et al.
Pubblicazione: (2025)

Diffusion Language Models Are Natively Length-Aware
di: Rossi, Vittorio, et al.
Pubblicazione: (2026)

On the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languages
di: Terzić, Aleksandar, et al.
Pubblicazione: (2024)

Confidence Regularized Masked Language Modeling using Text Length
di: Ji, Seunghyun, et al.
Pubblicazione: (2025)

From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers
di: Duan, Shaoxiong, et al.
Pubblicazione: (2023)

Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacements
di: Zhao, Guangxiang, et al.
Pubblicazione: (2025)

Context-Aware Initialization for Reducing Generative Path Length in Diffusion Language Models
di: Miao, Tongyuan, et al.
Pubblicazione: (2025)

Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum
di: Pouransari, Hadi, et al.
Pubblicazione: (2024)

Rethinking Perplexity: Revealing the Impact of Input Length on Perplexity Evaluation in LLMs
di: Cheng, Letian, et al.
Pubblicazione: (2026)

Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
di: Liu, Jie, et al.
Pubblicazione: (2024)

A Minimum Description Length Approach to Regularization in Neural Networks
di: Abudy, Matan, et al.
Pubblicazione: (2025)

On Provable Length and Compositional Generalization
di: Ahuja, Kartik, et al.
Pubblicazione: (2024)

Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
di: Liu, Hengchang, et al.
Pubblicazione: (2026)

SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
di: Huang, Kaixuan, et al.
Pubblicazione: (2024)

Length-MAX Tokenizer for Language Models
di: Dong, Dong, et al.
Pubblicazione: (2025)

Variable-Length Semantic IDs for Recommender Systems
di: Khrylchenko, Kirill
Pubblicazione: (2026)

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
di: Hu, Chenzhi, et al.
Pubblicazione: (2026)

Length Desensitization in Direct Preference Optimization
di: Liu, Wei, et al.
Pubblicazione: (2024)

Hansel: Output Length Controlling Framework for Large Language Models
di: Song, Seoha, et al.
Pubblicazione: (2024)

Intrinsic Entropy of Context Length Scaling in LLMs
di: Shi, Jingzhe, et al.
Pubblicazione: (2025)

Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
di: Luo, Feng, et al.
Pubblicazione: (2026)

On the Optimal Reasoning Length for RL-Trained Language Models
di: Nohara, Daisuke, et al.
Pubblicazione: (2026)

Length Generalization Bounds for Transformers
di: Yang, Andy, et al.
Pubblicazione: (2026)

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
di: Cho, Hanseul, et al.
Pubblicazione: (2024)

Provable Length Generalization in Sequence Prediction via Spectral Filtering
di: Marsden, Annie, et al.
Pubblicazione: (2024)

The Role of Sparsity for Length Generalization in Transformers
di: Golowich, Noah, et al.
Pubblicazione: (2025)

Learning Variable-Length Tokenization for Generative Recommendation
di: Wang, Minhao, et al.
Pubblicazione: (2026)

Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling
di: Cai, Jianfeng, et al.
Pubblicazione: (2025)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
di: Ma, Xuezhe, et al.
Pubblicazione: (2024)

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
di: Wang, Xindi, et al.
Pubblicazione: (2024)

Unlocking the Potentials of Retrieval-Augmented Generation for Diffusion Language Models
di: Yu, Chuanyue, et al.
Pubblicazione: (2026)

Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training
di: Tran, Toan, et al.
Pubblicazione: (2025)

Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
di: Wei, Linye, et al.
Pubblicazione: (2025)

Transformers Can Achieve Length Generalization But Not Robustly
di: Zhou, Yongchao, et al.
Pubblicazione: (2024)

Explaining Length Bias in LLM-Based Preference Evaluations
di: Hu, Zhengyu, et al.
Pubblicazione: (2024)

LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
di: Wei, Songtao, et al.
Pubblicazione: (2026)

Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks
di: Sabbaghi, Mahdi, et al.
Pubblicazione: (2024)

Gecko: An Efficient Neural Architecture Inherently Processing Sequences with Arbitrary Lengths
di: Ma, Xuezhe, et al.
Pubblicazione: (2026)

Parallelizing Linear Transformers with the Delta Rule over Sequence Length
di: Yang, Songlin, et al.
Pubblicazione: (2024)