Cover Image

Predicting the Order of Upcoming Tokens Improves Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zuhri, Zayd M. K., Fuadi, Erland Hilman, Aji, Alham Fikri
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax
by: Zuhri, Zayd M. K., et al.
Published: (2025)

MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding
by: Zuhri, Zayd Muhammad Kawakibi, et al.
Published: (2024)

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
by: Ananta, Moses, et al.
Published: (2025)

LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation
by: Irawan, Patrick Amadeus, et al.
Published: (2026)

COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances
by: Wibowo, Haryo Akbarianto, et al.
Published: (2023)

COPUS: Co-adaptive Parallelism and Batch Size Selection in Large Language Model Training
by: Sakip, Akhmed, et al.
Published: (2026)

Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models
by: Susanto, Lucky, et al.
Published: (2026)

Beyond Transfer Accuracy: Faithful Circuits for Controlled Low-Resource Adaptation
by: Nur'aini, Khumaisa, et al.
Published: (2026)

Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning
by: Attia, Ahmed, et al.
Published: (2026)

LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages
by: Aji, Alham Fikri, et al.
Published: (2025)

Unveiling the Influence of Amplifying Language-Specific Neurons
by: Rahmanisa, Inaya, et al.
Published: (2025)

Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
by: Lyu, Chenyang, et al.
Published: (2024)

Balanced Multi-Factor In-Context Learning for Multilingual Large Language Models
by: Kaneko, Masahiro, et al.
Published: (2025)

Daisy-TTS: Simulating Wider Spectrum of Emotions via Prosody Embedding Decomposition
by: Chevi, Rendi, et al.
Published: (2024)

NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts
by: Adilazuarda, Muhammad Farid, et al.
Published: (2025)

CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections
by: Imam, Mohamed Fazli, et al.
Published: (2024)

Extracting General-use Transformers for Low-resource Languages via Knowledge Distillation
by: Cruz, Jan Christian Blaise, et al.
Published: (2025)

Provable Robustness Against a Union of $\ell_0$ Adversarial Attacks
by: Hammoudeh, Zayd, et al.
Published: (2023)

Training Data Influence Analysis and Estimation: A Survey
by: Hammoudeh, Zayd, et al.
Published: (2022)

LLM Olympiad: Why Model Evaluation Needs a Sealed Exam
by: Cruz, Jan Christian Blaise, et al.
Published: (2026)

Language-Specific Latent Process Hinders Cross-Lingual Performance
by: Lim, Zheng Wei, et al.
Published: (2025)

Data Laundering: Artificially Boosting Benchmark Results through Knowledge Distillation
by: Mansurov, Jonibek, et al.
Published: (2024)

Sense Representations Are Inducible Interfaces
by: Cruz, Jan Christian Blaise, et al.
Published: (2026)

Self-Distillation Improves DNA Sequence Inference
by: Yu, Tong, et al.
Published: (2024)

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
by: Shen, Yangyi, et al.
Published: (2026)

TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
by: Hudi, Frederikus, et al.
Published: (2025)

How Individual Traits and Language Styles Shape Preferences In Open-ended User-LLM Interaction: A Preliminary Study
by: Chevi, Rendi, et al.
Published: (2025)

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning
by: Elshabrawy, Ahmed, et al.
Published: (2024)

Can Multimodal LLMs do Visual Temporal Understanding and Reasoning? The answer is No!
by: Imam, Mohamed Fazli, et al.
Published: (2025)

The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation
by: Wibowo, Haryo Akbarianto, et al.
Published: (2024)

Crosslingual Reasoning through Test-Time Scaling
by: Yong, Zheng-Xin, et al.
Published: (2025)

Parallel Token Prediction for Language Models
by: Draxler, Felix, et al.
Published: (2025)

Improving Token-Based World Models with Parallel Observation Prediction
by: Cohen, Lior, et al.
Published: (2024)

Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
by: Qorib, Muhammad Reza, et al.
Published: (2024)

Traveling Salesman-Based Token Ordering Improves Stability in Homomorphically Encrypted Language Models
by: Rho, Donghwan, et al.
Published: (2025)

Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition
by: Basharin, Artem, et al.
Published: (2024)

Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
by: Jaquier, Noémie, et al.
Published: (2023)

From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction
by: Zhu, Mingcheng, et al.
Published: (2026)

Efficient Temporal Tokenization for Mobility Prediction with Large Language Models
by: He, Haoyu, et al.
Published: (2025)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
by: Wu, Zihao, et al.
Published: (2026)