Cover Image

On the Effect of (Near) Duplicate Subwords in Language Modelling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Schäfer, Anton, Hofmann, Thomas, Schlag, Imanol, Pimentel, Tiago
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments
by: Schäfer, Anton, et al.
Published: (2024)

Subword-Based Comparative Linguistics across 242 Languages Using Wikipedia Glottosets
by: Chelombitko, Iaroslav, et al.
Published: (2026)

Engineering A Large Language Model From Scratch
by: Oketunji, Abiodun Finbarrs
Published: (2024)

Large Language Model (LLM) Bias Index -- LLMBI
by: Oketunji, Abiodun Finbarrs, et al.
Published: (2023)

Memory Bank Compression for Continual Adaptation of Large Language Models
by: Katraouras, Thomas, et al.
Published: (2026)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
by: Oketunji, Abiodun Finbarrs
Published: (2023)

Ensemble Language Models for Multilingual Sentiment Analysis
by: Hasan, Md Arid
Published: (2024)

RBCorr: Response Bias Correction in Language Models
by: Bhatt, Om, et al.
Published: (2026)

Compact Example-Based Explanations for Language Models
by: Schoenegger, Loris, et al.
Published: (2026)

The Open Source Advantage in Large Language Models (LLMs)
by: Manchanda, Jiya, et al.
Published: (2024)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
by: Kalajdzievski, Damjan
Published: (2024)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
by: Ye, Xinwu, et al.
Published: (2025)

Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation
by: Stacey, Joe, et al.
Published: (2023)

MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering
by: Ness, Robert Osazuwa, et al.
Published: (2024)

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge
by: Chowdhury, Nafis, et al.
Published: (2025)

Language Models Are Implicitly Continuous
by: Marro, Samuele, et al.
Published: (2025)

Where Should LoRA Go? Component-Type Placement in Hybrid Language Models
by: Borobia, Hector, et al.
Published: (2026)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)

Context-Aware Clustering using Large Language Models
by: Tipirneni, Sindhu, et al.
Published: (2024)

Combining Language and Topic Models for Hierarchical Text Classification
by: Toit, Jaco du, et al.
Published: (2025)

Constraint-Driven Small Language Models Based on Agent and OpenAlex Knowledge Graph: Mining Conceptual Pathways and Discovering Innovation Points in Academic Papers
by: Xia, Ziye, et al.
Published: (2025)

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment
by: Jaber, Jaber, et al.
Published: (2026)

Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encoding
by: Zhu, Jiajun, et al.
Published: (2025)

Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
by: Shravan, Rohan
Published: (2026)

How Language Models Process Out-of-Distribution Inputs: A Two-Pathway Framework
by: Saghir, Hamidreza
Published: (2026)

Efficacy of ByT5 in Multilingual Translation of Biblical Texts for Underrepresented Languages
by: Aars, Corinne, et al.
Published: (2024)

Linguistically-Informed Multilingual Instruction Tuning: Is There an Optimal Set of Languages to Tune?
by: Soykan, Gürkan, et al.
Published: (2024)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
by: Özeren, Enes, et al.
Published: (2025)

Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning
by: Gao, Yilin, et al.
Published: (2024)

Neural Activation Patterns Across Language Model Architectures: A Comprehensive Analysis of Cognitive Task Performance
by: Naser-Moghadasi, Mahdi, et al.
Published: (2026)

SampoNLP: A Self-Referential Toolkit for Morphological Analysis of Subword Tokenizers
by: Chelombitko, Iaroslav, et al.
Published: (2026)

KenSwQuAD -- A Question Answering Dataset for Swahili Low Resource Language
by: Wanjawa, Barack W., et al.
Published: (2022)

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction
by: Walker, Nicholas
Published: (2024)

Identifying and Mitigating the Influence of the Prior Distribution in Large Language Models
by: Zhang, Liyi, et al.
Published: (2025)

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding
by: Feng, Qi, et al.
Published: (2025)

Better and Worse with Scale: How Contextual Entrainment Diverges with Model Size
by: Kukreja, Dikshant, et al.
Published: (2026)

Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs
by: Sun, Luoyang, et al.
Published: (2026)

Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
by: Hanna, Michael, et al.
Published: (2024)

BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Data
by: Tastet, Jean-Loup, et al.
Published: (2024)