Copertina

Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Balde, Gunjan, Roy, Soumyadeep, Mondal, Mainack, Ganguly, Niloy
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Learning Faster with Better Tokens: Parameter-Efficient Vocabulary Adaptation for Specialized Text Summarization
di: Balde, Gunjan, et al.
Pubblicazione: (2026)

MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarization
di: Balde, Gunjan, et al.
Pubblicazione: (2024)

Evaluation of LLMs in Medical Text Summarization: The Role of Vocabulary Adaptation in High OOV Settings
di: Balde, Gunjan, et al.
Pubblicazione: (2025)

Unlocking Efficiency: Adaptive Masking for Gene Transformer Models
di: Roy, Soumyadeep, et al.
Pubblicazione: (2024)

BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training
di: Chizhov, Pavel, et al.
Pubblicazione: (2024)

BlockBPE: Parallel BPE Tokenization
di: You, Amos
Pubblicazione: (2025)

Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
di: Roy, Soumyadeep, et al.
Pubblicazione: (2024)

An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
di: Cognetta, Marco, et al.
Pubblicazione: (2024)

Batching BPE Tokenization Merges
di: Morgan, Alexander P.
Pubblicazione: (2024)

Constructing a BPE Tokenization DFA
di: Berglund, Martin, et al.
Pubblicazione: (2024)

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
di: Lian, Haoran, et al.
Pubblicazione: (2024)

Byte BPE Tokenization as an Inverse string Homomorphism
di: Geng, Saibo, et al.
Pubblicazione: (2024)

Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes
di: Pleimling, Xavier, et al.
Pubblicazione: (2026)

LiteToken: Removing Intermediate Merge Residues From BPE Tokenizers
di: Sun, Yike, et al.
Pubblicazione: (2026)

From Characters to Tokens: Dynamic Grouping with Hierarchical BPE
di: Dolga, Rares, et al.
Pubblicazione: (2025)

AdaptBPE: From General Purpose to Specialized Tokenizers
di: Liyanage, Vijini, et al.
Pubblicazione: (2026)

Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLP
di: Visser, Ruan, et al.
Pubblicazione: (2026)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
di: Kalamkar, Prathamesh, et al.
Pubblicazione: (2025)

Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models
di: Alrefaie, Mohamed Taher, et al.
Pubblicazione: (2024)

Morphological Typology in BPE Subword Productivity and Language Modeling
di: Parra, Iñigo
Pubblicazione: (2024)

SuperBPE: Space Travel for Language Models
di: Liu, Alisa, et al.
Pubblicazione: (2025)

Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs
di: Nag, Arijit, et al.
Pubblicazione: (2024)

Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation
di: Moroni, Luca, et al.
Pubblicazione: (2025)

Small Vocabularies, Big Gains: Pretraining and Tokenization in Time Series Models
di: Roger, Alexis, et al.
Pubblicazione: (2025)

Evaluating Subword Tokenization Techniques for Bengali: A Benchmark Study with BengaliBPE
di: Patwary, Firoj Ahmmed, et al.
Pubblicazione: (2025)

GPUTOK: GPU Accelerated Byte Level BPE Tokenization
di: Kadamba, Venu Gopal, et al.
Pubblicazione: (2026)

Geographic Adaptation of Pretrained Language Models
di: Hofmann, Valentin, et al.
Pubblicazione: (2022)

TokAlign: Efficient Vocabulary Adaptation via Token Alignment
di: Li, Chong, et al.
Pubblicazione: (2025)

Efficient Continual Pre-training of LLMs for Low-resource Languages
di: Nag, Arijit, et al.
Pubblicazione: (2024)

Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models
di: Sawada, Tomohiro, et al.
Pubblicazione: (2025)

Knowledge of Pretrained Language Models on Surface Information of Tokens
di: Hiraoka, Tatsuya, et al.
Pubblicazione: (2024)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
di: Xie, Wanyun, et al.
Pubblicazione: (2025)

Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models
di: Poddar, Soham, et al.
Pubblicazione: (2025)

TokAlign++: Advancing Vocabulary Adaptation via Better Token Alignment
di: Li, Chong, et al.
Pubblicazione: (2026)

Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment
di: Vemula, Saketh Reddy, et al.
Pubblicazione: (2025)

A Tutorial on the Pretrain-Finetune Paradigm for Natural Language Processing
di: Wang, Yu, et al.
Pubblicazione: (2024)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
di: Somayajula, Sai Ashish, et al.
Pubblicazione: (2024)

Pretraining with Token-Level Adaptive Latent Chain-of-Thought
di: Zeng, Boyi, et al.
Pubblicazione: (2026)

Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers
di: Yoshida, Davis
Pubblicazione: (2024)

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?
di: Hayase, Jonathan, et al.
Pubblicazione: (2024)