Imagen de Portada

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kalamkar, Prathamesh, Letcher, Ned, Chami, Meissane, Lad, Sahger, Mohanty, Shayan, Pendse, Prasanna
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Towards Transparent AI Grading: Semantic Entropy as a Signal for Human-AI Disagreement
por: Iyer, Karrtik, et al.
Publicado: (2025)

Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models
por: Balde, Gunjan, et al.
Publicado: (2024)

How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them
por: Liao, Disen, et al.
Publicado: (2026)

Indian Legal NLP Benchmarks : A Survey
por: Kalamkar, Prathamesh, et al.
Publicado: (2021)

Small Vocabularies, Big Gains: Pretraining and Tokenization in Time Series Models
por: Roger, Alexis, et al.
Publicado: (2025)

Aalap: AI Assistant for Legal & Paralegal Functions in India
por: Tiwari, Aman, et al.
Publicado: (2024)

Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems
por: Devadiga, Prathamesh
Publicado: (2025)

Knowledge of Pretrained Language Models on Surface Information of Tokens
por: Hiraoka, Tatsuya, et al.
Publicado: (2024)

Making Large Language Models Speak Tulu: Structured Prompting for an Extremely Low-Resource Language
por: Devadiga, Prathamesh, et al.
Publicado: (2026)

Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models
por: Alrefaie, Mohamed Taher, et al.
Publicado: (2024)

SOI Matters: Analyzing Multi-Setting Training Dynamics in Pretrained Language Models via Subsets of Interest
por: Vassef, Shayan, et al.
Publicado: (2025)

Token-level Ensembling of Models with Different Vocabularies
por: Wicks, Rachel, et al.
Publicado: (2025)

Large Vocabulary Size Improves Large Language Models
por: Takase, Sho, et al.
Publicado: (2024)

Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
por: Peng, Letian, et al.
Publicado: (2024)

Towards Linguistically-Aware and Language-Independent Tokenization for Large Language Models (LLMs)
por: Rahman, Abrar, et al.
Publicado: (2024)

Exploring the Benefits of Domain-Pretraining of Generative Large Language Models for Chemistry
por: Acharya, Anurag, et al.
Publicado: (2024)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
por: Soldaini, Luca, et al.
Publicado: (2024)

Gradient Localization Improves Lifelong Pretraining of Language Models
por: Fernandez, Jared, et al.
Publicado: (2024)

Phonological Representation Learning for Isolated Signs Improves Out-of-Vocabulary Generalization
por: Kezar, Lee, et al.
Publicado: (2025)

On Linear Representations and Pretraining Data Frequency in Language Models
por: Merullo, Jack, et al.
Publicado: (2025)

Gender Encoding Patterns in Pretrained Language Model Representations
por: Zakizadeh, Mahdi, et al.
Publicado: (2025)

Review Based Entity Ranking using Fuzzy Logic Algorithmic Approach: Analysis
por: Kalamkar, Pratik N., et al.
Publicado: (2025)

Exploring the Effect of Segmentation and Vocabulary Size on Speech Tokenization for Speech Language Models
por: Kando, Shunsuke, et al.
Publicado: (2025)

Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models
por: Jiang, Peihai, et al.
Publicado: (2025)

Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation
por: Liu, Yifan, et al.
Publicado: (2025)

Implicit Self-supervised Language Representation for Spoken Language Diarization
por: Mishra, Jagabandhu, et al.
Publicado: (2023)

How Do Different Forms of Note‐Taking Affect Second Language Vocabulary Learning?
por: Zhouhan Jin, et al.
Publicado: (2025)

The Role of Vocabularies in Learning Sparse Representations for Ranking
por: Kim, Hiun, et al.
Publicado: (2025)

Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour
por: Norris, Max, et al.
Publicado: (2025)

Code Pretraining Improves Entity Tracking Abilities of Language Models
por: Kim, Najoung, et al.
Publicado: (2024)

The Geometry of Tokens in Internal Representations of Large Language Models
por: Viswanathan, Karthik, et al.
Publicado: (2025)

Improved Representation Steering for Language Models
por: Wu, Zhengxuan, et al.
Publicado: (2025)

Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling
por: Zhang, Zhen, et al.
Publicado: (2026)

Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets
por: Barbu, Eduard, et al.
Publicado: (2025)

Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning
por: Kim, Nayeon, et al.
Publicado: (2025)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
por: Zhang, Zihou, et al.
Publicado: (2026)

How Does Code Pretraining Affect Language Model Task Performance?
por: Petty, Jackson, et al.
Publicado: (2024)

LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations
por: Gottesman, Daniela, et al.
Publicado: (2025)

Data Caricatures: On the Representation of African American Language in Pretraining Corpora
por: Deas, Nicholas, et al.
Publicado: (2025)

"It's how you do things that matters": Attending to Process to Better Serve Indigenous Communities with Language Technologies
por: Cooper, Ned, et al.
Publicado: (2024)