Cover Image

Language Models Without a Trainable Input Embedding Table: Learning from Fixed Minimal Binary Token Codes

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Bochkov, A.
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
by: Bochkov, A.
Published: (2025)

Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
by: Bochkov, A.
Published: (2025)

Token Distillation: Attention-aware Input Embeddings For New Tokens
by: Dobler, Konstantin, et al.
Published: (2025)

Probabilistic Calibration Is a Trainable Capability in Language Models
by: Baldelli, Davide, et al.
Published: (2026)

Estimating Knowledge in Large Language Models Without Generating a Single Token
by: Gottesman, Daniela, et al.
Published: (2024)

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data
by: Xia, Han, et al.
Published: (2024)

Enhancing Multilingual Language Models for Code-Switched Input Data
by: Xie, Katherine, et al.
Published: (2025)

Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models
by: Qiu, Junxiang, et al.
Published: (2026)

LLM4Decompile: Decompiling Binary Code with Large Language Models
by: Tan, Hanzhuo, et al.
Published: (2024)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
by: Cho, Gyeongje, et al.
Published: (2025)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
by: Cao, Qingqing, et al.
Published: (2023)

DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
by: Wu, Zirui, et al.
Published: (2026)

Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language
by: Kumar, Vinayshekhar Bannihatti, et al.
Published: (2026)

Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models
by: Zhang, Ying, et al.
Published: (2024)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
by: Levy, Mosh, et al.
Published: (2024)

ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training
by: Han, Feijiang, et al.
Published: (2025)

Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Models
by: Kukić, Matija Luka, et al.
Published: (2026)

Language Modeling with Learned Meta-Tokens
by: Shah, Alok N., et al.
Published: (2025)

Trainable Transformer in Transformer
by: Panigrahi, Abhishek, et al.
Published: (2023)

Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
by: Xu, Zihao, et al.
Published: (2024)

How Different Tokenization Algorithms Impact LLMs and Transformer Models for Binary Code Analysis
by: Mostafa, Ahmed, et al.
Published: (2025)

TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
by: Tseng, Liang-Hsuan, et al.
Published: (2026)

Problematic Tokens: Tokenizer Bias in Large Language Models
by: Yang, Jin, et al.
Published: (2024)

TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
by: Tseng, Liang-Hsuan, et al.
Published: (2025)

Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models
by: Xu, Zihao, et al.
Published: (2026)

Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
by: Yang, Jinbiao
Published: (2024)

Learning Code-Edit Embedding to Model Student Debugging Behavior
by: Heickal, Hasnain, et al.
Published: (2025)

Transfer Learning with Clinical Concept Embeddings from Large Language Models
by: Gao, Yuhe, et al.
Published: (2024)

Memory Tokens: Large Language Models Can Generate Reversible Sentence Embeddings
by: Sastre, Ignacio, et al.
Published: (2025)

Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models
by: Zhang, Christine, et al.
Published: (2026)

Subword Tokenization Strategies for Kurdish Word Embeddings
by: Salehi, Ali, et al.
Published: (2025)

One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
by: Zhu, Yutao, et al.
Published: (2024)

Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs
by: Yaldiz, Duygu Nur, et al.
Published: (2024)

Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
by: Zheng, Brian Siyuan, et al.
Published: (2025)

Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning
by: Ye, Ziang, et al.
Published: (2024)

Characterizing the Expressivity of Fixed-Precision Transformer Language Models
by: Li, Jiaoda, et al.
Published: (2025)

TALL -- A Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages
by: Ofer, Moshe, et al.
Published: (2025)

Kathleen: Oscillator-Based Byte-Level Text Classification Without Tokenization or Attention
by: Fountzoulas, George
Published: (2026)

SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings
by: SadraeiJavaeri, MohammadAli, et al.
Published: (2024)