Imagen de Portada

NITP: Next Implicit Token Prediction for LLM Pre-training

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Xiangdong, Zhang, Debing, Zhang, Shaofeng, Qin, Xiaohan, Cheng, Yu, Yan, Junchi
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views
por: Zhang, Xiangdong, et al.
Publicado: (2025)

PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders
por: Zhang, Xiangdong, et al.
Publicado: (2024)

Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge
por: Tang, Pengwei, et al.
Publicado: (2025)

Reasoning Bias of Next Token Prediction Training
por: Lin, Pengxiao, et al.
Publicado: (2025)

Implicit Optimization Bias of Next-Token Prediction in Linear Models
por: Thrampoulidis, Christos
Publicado: (2024)

Cautious Next Token Prediction
por: Wang, Yizhou, et al.
Publicado: (2025)

Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
por: Tian, Junfeng, et al.
Publicado: (2024)

Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations
por: Zhao, Yize, et al.
Publicado: (2025)

Token Prediction as Implicit Classification to Identify LLM-Generated Text
por: Chen, Yutian, et al.
Publicado: (2023)

Anatomical Structure-Guided Medical Vision-Language Pre-training
por: Li, Qingqiu, et al.
Publicado: (2024)

Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models
por: Zhang, Ying, et al.
Publicado: (2024)

Dual-Branch Center-Surrounding Contrast: Rethinking Contrastive Learning for 3D Point Clouds
por: Zhang, Shaofeng, et al.
Publicado: (2025)

SENTRA: Selected-Next-Token Transformer for LLM Text Detection
por: Plyler, Mitchell, et al.
Publicado: (2025)

ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning
por: Qin, Xiaohan, et al.
Publicado: (2025)

Next Token Perception Score: Analytical Assessment of your LLM Perception Skills
por: Cheng, Yu-Ang, et al.
Publicado: (2025)

QoSBERT: An Uncertainty-Aware Approach based on Pre-trained Language Models for Service Quality Prediction
por: Wang, Ziliang, et al.
Publicado: (2025)

Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens
por: Zhang, Anqi, et al.
Publicado: (2024)

VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models
por: Zhang, Xiangdong, et al.
Publicado: (2025)

Scaling LLM Pre-training with Vocabulary Curriculum
por: Yu, Fangyuan
Publicado: (2025)

Diversity or Precision? A Deep Dive into Next Token Prediction
por: Wu, Haoyuan, et al.
Publicado: (2025)

Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pre-trained Models
por: Purason, Taido, et al.
Publicado: (2025)

NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
por: Han, Minglun, et al.
Publicado: (2024)

On Predicting the Post-training Potential of Pre-trained LLMs
por: Li, Xiaoyuan, et al.
Publicado: (2026)

JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation
por: Yang, Yebin, et al.
Publicado: (2026)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
por: Zhang, Haode, et al.
Publicado: (2022)

Training LLMs Beyond Next Token Prediction -- Filling the Mutual Information Gap
por: Yang, Chun-Hao, et al.
Publicado: (2025)

ENTP: Encoder-only Next Token Prediction
por: Ewer, Ethan, et al.
Publicado: (2024)

GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
por: Xia, Renqiu, et al.
Publicado: (2024)

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
por: Yan, Junbing, et al.
Publicado: (2024)

Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs?
por: Zeng, Hansi, et al.
Publicado: (2025)

Next-Token Prediction Task Assumes Optimal Data Ordering for LLM Training in Proof Generation
por: An, Chenyang, et al.
Publicado: (2024)

New Intent Discovery with Pre-training and Contrastive Learning
por: Zhang, Yuwei, et al.
Publicado: (2022)

Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
por: Wang, Xixi, et al.
Publicado: (2025)

Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training
por: Zhang, Longhui, et al.
Publicado: (2024)

Text-to-Code Generation with Modality-relative Pre-training
por: Christopoulou, Fenia, et al.
Publicado: (2024)

From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs
por: Wang, Shaojie, et al.
Publicado: (2026)

Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
por: Jin, Yang, et al.
Publicado: (2024)

Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension
por: Qi, Mengnan, et al.
Publicado: (2024)

SoftMCL: Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training
por: Wang, Jin, et al.
Publicado: (2024)

Effectiveness of Pre-training for Few-shot Intent Classification
por: Zhang, Haode, et al.
Publicado: (2021)