Cover Image

PonderLM: Pretraining Language Models to Ponder in Continuous Space

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Boyi, Song, Shixiang, Huang, Siyuan, Wang, Yixuan, Li, He, He, Ziwei, Wang, Xinbing, Li, Zhiyu, Lin, Zhouhan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
by: Zeng, Boyi, et al.
Published: (2025)

PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Masking
by: Li, He, et al.
Published: (2026)

AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth
by: Song, Shixiang, et al.
Published: (2026)

Learning to Ponder: Adaptive Reasoning in Latent Space
by: He, Yixin, et al.
Published: (2025)

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
by: Zeng, Boyi, et al.
Published: (2025)

Pretraining with Token-Level Adaptive Latent Chain-of-Thought
by: Zeng, Boyi, et al.
Published: (2026)

FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
by: Kai, Jushi, et al.
Published: (2025)

HuRef: HUman-REadable Fingerprint for Large Language Models
by: Zeng, Boyi, et al.
Published: (2023)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
by: Shen, Junzhe, et al.
Published: (2026)

FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
by: Zhang, Runzhe, et al.
Published: (2026)

Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Models
by: Liu, Yuliang, et al.
Published: (2026)

Context-level Language Modeling by Learning Predictive Context Embeddings
by: Dai, Beiya, et al.
Published: (2025)

Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
by: He, Ziwei, et al.
Published: (2023)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
by: Cao, Jiaqi, et al.
Published: (2025)

KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning
by: Yu, Peng, et al.
Published: (2024)

KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models
by: Chen, Zirui, et al.
Published: (2025)

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
by: Wang, Yixuan, et al.
Published: (2026)

When to Ponder: Adaptive Compute Allocation for Code Generation via Test-Time Training
by: Sim, Gihyeon
Published: (2025)

Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment
by: Cai, Zhuoxuan, et al.
Published: (2025)

Mirror-Consistency: Harnessing Inconsistency in Majority Voting
by: Huang, Siyuan, et al.
Published: (2024)

Dynamic Pondering Sparsity-aware Mixture-of-Experts Transformer for Event Stream based Visual Object Tracking
by: Wang, Shiao, et al.
Published: (2026)

Leveraging Grammar Induction for Language Understanding and Generation
by: Kai, Jushi, et al.
Published: (2024)

Foundation CAN LM: A Pretrained Language Model For Automotive CAN Data
by: Esashi, Akiharu, et al.
Published: (2026)

CogLM: Tracking Cognitive Development of Large Language Models
by: Wang, Xinglin, et al.
Published: (2024)

From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models
by: He, Kaiyu, et al.
Published: (2025)

MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining
by: Dong, Haoyu, et al.
Published: (2025)

Towards Controlled Table-to-Text Generation with Scientific Reasoning
by: Guo, Zhixin, et al.
Published: (2023)

Critical Data Size of Language Models from a Grokking Perspective
by: Zhu, Xuekai, et al.
Published: (2024)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
by: Lu, Liming, et al.
Published: (2026)

ControlLM: Crafting Diverse Personalities for Language Models
by: Weng, Yixuan, et al.
Published: (2024)

Reverse-Engineering Model Editing on Language Models
by: Sun, Zhiyu, et al.
Published: (2026)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

MLP Memory: A Retriever-Pretrained Memory for Large Language Models
by: Wei, Rubin, et al.
Published: (2025)

Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets
by: Xue, Bo, et al.
Published: (2026)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
by: Zhou, Ying, et al.
Published: (2024)

Static Program Slicing Using Language Models With Dataflow-Aware Pretraining and Constrained Decoding
by: He, Pengfei, et al.
Published: (2026)

ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models
by: Guo, Wenbin, et al.
Published: (2025)

Graph Parsing Networks
by: Song, Yunchong, et al.
Published: (2024)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

Xmodel-LM Technical Report
by: Wang, Yichuan, et al.
Published: (2024)