Imagen de Portada

Physics in Next-token Prediction

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	An, Hongjun, Song, Yiliang, Li, Xuelong
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

CreditAudit: 2$^\text{nd}$ Dimension for LLM Evaluation and Selection
por: Song, Yiliang, et al.
Publicado: (2026)

Next-token pretraining implies in-context learning
por: Riechers, Paul M., et al.
Publicado: (2025)

Interpretable Next-token Prediction via the Generalized Induction Head
por: Kim, Eunji, et al.
Publicado: (2024)

SUN: Shared Use of Next-token Prediction for Efficient Multi-LLM Disaggregated Serving
por: Woo, Sunghyeon, et al.
Publicado: (2026)

Not all tokens are needed(NAT): token efficient reinforcement learning
por: Sang, Hejian, et al.
Publicado: (2026)

All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling
por: Marconato, Emanuele, et al.
Publicado: (2024)

Theoretical Foundations of Scaling Law in Familial Models
por: Song, Huan, et al.
Publicado: (2025)

Single-Pixel Vision-Language Model for Intrinsic Privacy-Preserving Behavioral Intelligence
por: An, Hongjun, et al.
Publicado: (2026)

Scaling FP8 training to trillion-token LLMs
por: Fishman, Maxim, et al.
Publicado: (2024)

An attempt to generate new bridge types from latent space of energy-based model
por: Zhang, Hongjun
Publicado: (2024)

The pitfalls of next-token prediction
por: Bachmann, Gregor, et al.
Publicado: (2024)

Looking beyond the next token
por: Thankaraj, Abitha, et al.
Publicado: (2025)

Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Models
por: Xue, Siqiao, et al.
Publicado: (2024)

Are LLMs Vulnerable to Preference-Undermining Attacks (PUA)? A Factorial Analysis Methodology for Diagnosing the Trade-off between Preference Alignment and Real-World Validity
por: An, Hongjun, et al.
Publicado: (2026)

SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
por: An, Hongjun, et al.
Publicado: (2024)

GReF: A Unified Generative Framework for Efficient Reranking via Ordered Multi-token Prediction
por: Lin, Zhijie, et al.
Publicado: (2025)

Deep Contrastive Graph Learning with Clustering-Oriented Guidance
por: Chen, Mulin, et al.
Publicado: (2024)

Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One
por: Zhang, Hongyuan, et al.
Publicado: (2023)

Resilience Inference for Supply Chains with Hypergraph Neural Network
por: Shen, Zetian, et al.
Publicado: (2025)

Bio2Token: All-atom tokenization of any biomolecular structure with Mamba
por: Liu, Andrew, et al.
Publicado: (2024)

Shaping capabilities with token-level data filtering
por: Rathi, Neil, et al.
Publicado: (2026)

Provable Long-Range Benefits of Next-Token Prediction
por: Cao, Xinyuan, et al.
Publicado: (2025)

Next Embedding Prediction Makes World Models Stronger
por: Bredis, George, et al.
Publicado: (2026)

Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL
por: Sokar, Ghada, et al.
Publicado: (2024)

Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarks
por: Song, Yiliang, et al.
Publicado: (2026)

Leveraging Taxonomy Similarity for Next Activity Prediction in Patient Treatment
por: Kuhn, Martin, et al.
Publicado: (2025)

Scaling Transformer to 1M tokens and beyond with RMT
por: Bulatov, Aydar, et al.
Publicado: (2023)

Towards Learnable Anchor for Deep Multi-View Clustering
por: Wang, Bocheng, et al.
Publicado: (2025)

Unveiling the Inflexibility of Adaptive Embedding in Traffic Forecasting
por: Wang, Hongjun, et al.
Publicado: (2024)

Cautious Next Token Prediction
por: Wang, Yizhou, et al.
Publicado: (2025)

An Accurate and Low-Parameter Machine Learning Architecture for Next Location Prediction
por: Jary, Calvin, et al.
Publicado: (2024)

Language models are better than humans at next-token prediction
por: Shlegeris, Buck, et al.
Publicado: (2022)

Can We Predict Your Next Move Without Breaking Your Privacy?
por: Soni, Arpita, et al.
Publicado: (2025)

Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
por: Gagnon, Leo, et al.
Publicado: (2025)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
por: Li, Junlin, et al.
Publicado: (2026)

You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
por: Xu, Yijie, et al.
Publicado: (2025)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
por: Kwek, Eugene, et al.
Publicado: (2025)

An Innovative Next Activity Prediction Using Process Entropy and Dynamic Attribute-Wise-Transformer in Predictive Business Process Monitoring
por: Zare, Hadi, et al.
Publicado: (2025)

Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
por: Huang, Zhuoxu, et al.
Publicado: (2026)

CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Prediction
por: Chen, Yuzhu, et al.
Publicado: (2025)