Cover Image

Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yuliang, Song, Yunchong, Wang, Yixuan, Ge, Kewen, Lamb, Alex, Guo, Qipeng, Chen, Kai, Zhou, Bowen, Lin, Zhouhan
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Context-level Language Modeling by Learning Predictive Context Embeddings
by: Dai, Beiya, et al.
Published: (2025)

MLP Memory: A Retriever-Pretrained Memory for Large Language Models
by: Wei, Rubin, et al.
Published: (2025)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
by: Cao, Jiaqi, et al.
Published: (2025)

PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
by: Zeng, Boyi, et al.
Published: (2025)

Critical Data Size of Language Models from a Grokking Perspective
by: Zhu, Xuekai, et al.
Published: (2024)

PonderLM: Pretraining Language Models to Ponder in Continuous Space
by: Zeng, Boyi, et al.
Published: (2025)

Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention
by: Huang, Siyuan, et al.
Published: (2024)

Leveraging Grammar Induction for Language Understanding and Generation
by: Kai, Jushi, et al.
Published: (2024)

Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
by: Dong, ZiYi, et al.
Published: (2026)

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
by: Wang, Yixuan, et al.
Published: (2026)

Towards Compressive and Scalable Recurrent Memory
by: Song, Yunchong, et al.
Published: (2026)

AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth
by: Song, Shixiang, et al.
Published: (2026)

How Do Different Forms of Note‐Taking Affect Second Language Vocabulary Learning?
by: Zhouhan Jin, et al.
Published: (2025)

PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning
by: Zhu, Xuekai, et al.
Published: (2023)

ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
by: Yang, Yuancheng, et al.
Published: (2026)

FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
by: Kai, Jushi, et al.
Published: (2025)

Graph Parsing Networks
by: Song, Yunchong, et al.
Published: (2024)

HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models
by: Hossain, Jawad, et al.
Published: (2026)

Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
by: Laitenberger, Alex, et al.
Published: (2025)

SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
by: Kai, Jushi, et al.
Published: (2024)

Evaluating and Adapting Large Language Models to Represent Folktales in Low-Resource Languages
by: Meaney, JA, et al.
Published: (2024)

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
by: Zeng, Boyi, et al.
Published: (2025)

Stronger Models are NOT Stronger Teachers for Instruction Tuning
by: Xu, Zhangchen, et al.
Published: (2024)

SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
by: Ma, Ming, et al.
Published: (2025)

Pretraining with Token-Level Adaptive Latent Chain-of-Thought
by: Zeng, Boyi, et al.
Published: (2026)

DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting
by: Lv, Kai, et al.
Published: (2025)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
by: Peng, Runyu, et al.
Published: (2026)

Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization
by: Lu, Junjie, et al.
Published: (2025)

How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data
by: Huang, Zixian, et al.
Published: (2026)

Efficient Joint Prediction of Multiple Future Tokens
by: Ahn, Kwangjun, et al.
Published: (2025)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
by: Shen, Junzhe, et al.
Published: (2026)

Full Parameter Fine-tuning for Large Language Models with Limited Resources
by: Lv, Kai, et al.
Published: (2023)

Data-free Weight Compress and Denoise for Large Language Models
by: Peng, Runyu, et al.
Published: (2024)

A Semantic Space is Worth 256 Language Descriptions: Make Stronger Segmentation Models with Descriptive Properties
by: Xiao, Junfei, et al.
Published: (2023)

MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables
by: Yeh, Sung-Lin, et al.
Published: (2026)

FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
by: Zhang, Runzhe, et al.
Published: (2026)

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
by: Gong, Zhuocheng, et al.
Published: (2025)

Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders
by: Zhang, Yingji, et al.
Published: (2024)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
by: Guo, Xu, et al.
Published: (2026)

Navigating the Concept Space of Language Models
by: Marcílio-Jr, Wilson E., et al.
Published: (2026)