Copertina

SpaceByte: Towards Deleting Tokenization from Large Language Modeling

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Slagle, Kevin
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
di: Kallini, Julie, et al.
Pubblicazione: (2024)

Counterfactual Token Generation in Large Language Models
di: Chatzi, Ivi, et al.
Pubblicazione: (2024)

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization
di: Foroutan, Negar, et al.
Pubblicazione: (2025)

Token-Efficient Leverage Learning in Large Language Models
di: Zeng, Yuanhao, et al.
Pubblicazione: (2024)

Compositional Steering of Large Language Models with Steering Tokens
di: Radevski, Gorjan, et al.
Pubblicazione: (2026)

Probabilistic Token Alignment for Large Language Model Fusion
di: Zeng, Runjia, et al.
Pubblicazione: (2025)

Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
di: Zhang, Zhongjian, et al.
Pubblicazione: (2026)

A Law of Next-Token Prediction in Large Language Models
di: He, Hangfeng, et al.
Pubblicazione: (2024)

Scalable Token-Level Hallucination Detection in Large Language Models
di: Min, Rui, et al.
Pubblicazione: (2026)

Evaluation of Large Language Models via Coupled Token Generation
di: Benz, Nina Corvelo, et al.
Pubblicazione: (2025)

MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling
di: Limisiewicz, Tomasz, et al.
Pubblicazione: (2024)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
di: Jia, Sheng, et al.
Pubblicazione: (2025)

Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
di: Cao, Qi, et al.
Pubblicazione: (2026)

TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
di: Choo, Jinho, et al.
Pubblicazione: (2026)

Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
di: Egli, Eric, et al.
Pubblicazione: (2025)

Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
di: Ding, Fangyu, et al.
Pubblicazione: (2026)

Large Language Models Struggle in Token-Level Clinical Named Entity Recognition
di: Lu, Qiuhao, et al.
Pubblicazione: (2024)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
di: Shao, Chenze, et al.
Pubblicazione: (2024)

Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
di: Mao, Liyuan, et al.
Pubblicazione: (2026)

Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models
di: Alqahtani, Sawsan, et al.
Pubblicazione: (2026)

DP-Fusion: Token-Level Differentially Private Inference for Large Language Models
di: Thareja, Rushil, et al.
Pubblicazione: (2025)

TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
di: Zhang, Tunyu, et al.
Pubblicazione: (2025)

Memory Tokens: Large Language Models Can Generate Reversible Sentence Embeddings
di: Sastre, Ignacio, et al.
Pubblicazione: (2025)

GPUTOK: GPU Accelerated Byte Level BPE Tokenization
di: Kadamba, Venu Gopal, et al.
Pubblicazione: (2026)

Demystifying Embedding Spaces using Large Language Models
di: Tennenholtz, Guy, et al.
Pubblicazione: (2023)

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models
di: Neitemeier, Pit, et al.
Pubblicazione: (2025)

Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
di: Fadeeva, Ekaterina, et al.
Pubblicazione: (2024)

Length-MAX Tokenizer for Language Models
di: Dong, Dong, et al.
Pubblicazione: (2025)

Towards Trustable Language Models: Investigating Information Quality of Large Language Models
di: Rejeleene, Rick, et al.
Pubblicazione: (2024)

Towards detecting unanticipated bias in Large Language Models
di: Kruspe, Anna
Pubblicazione: (2024)

Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
di: Leyton-Brown, Kevin, et al.
Pubblicazione: (2024)

Fast Byte Latent Transformer
di: Kallini, Julie, et al.
Pubblicazione: (2026)

Understanding and Mitigating Tokenization Bias in Language Models
di: Phan, Buu, et al.
Pubblicazione: (2024)

Adapting Language Models via Token Translation
di: Feng, Zhili, et al.
Pubblicazione: (2024)

Guiding Language Model Reasoning with Planning Tokens
di: Wang, Xinyi, et al.
Pubblicazione: (2023)

From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representation
di: Kiulian, Artur, et al.
Pubblicazione: (2024)

Interpreting the Repeated Token Phenomenon in Large Language Models
di: Yona, Itay, et al.
Pubblicazione: (2025)

Toward Adaptive Reasoning in Large Language Models with Thought Rollback
di: Chen, Sijia, et al.
Pubblicazione: (2024)

DistiLLM: Towards Streamlined Distillation for Large Language Models
di: Ko, Jongwoo, et al.
Pubblicazione: (2024)

FaithLM: Towards Faithful Explanations for Large Language Models
di: Chuang, Yu-Neng, et al.
Pubblicazione: (2024)