Imagen de Portada

Model-Aware Tokenizer Transfer

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Haltiuk, Mykola, Smywinski-Pohl, Aleksander
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Targum -- A Multilingual New Testament Translation Corpus
por: Rapacz, Maciej, et al.
Publicado: (2026)

Are manual annotations necessary for statutory interpretations retrieval?
por: Smywiński-Pohl, Aleksander, et al.
Publicado: (2025)

eFontes. Part of Speech Tagging and Lemmatization of Medieval Latin Texts.A Cross-Genre Survey
por: Nowak, Krzysztof, et al.
Publicado: (2024)

LLM-as-a-Judge is Bad, Based on AI Attempting the Exam Qualifying for the Member of the Polish National Board of Appeal
por: Karp, Michał, et al.
Publicado: (2025)

Just on Time: Token-Level Early Stopping for Diffusion Language Models
por: Kohut, Zahar, et al.
Publicado: (2026)

Zero-Shot Tokenizer Transfer
por: Minixhofer, Benjamin, et al.
Publicado: (2024)

On Adversarial Robustness of Language Models in Transfer Learning
por: Turbal, Bohdan, et al.
Publicado: (2024)

TASE: Token Awareness and Structured Evaluation for Multilingual Language Models
por: Zhao, Chenzhuo, et al.
Publicado: (2025)

SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
por: Liu, Dong, et al.
Publicado: (2025)

Everyone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data
por: Raes, Rik, et al.
Publicado: (2024)

Training Text-to-Molecule Models with Context-Aware Tokenization
por: Kim, Seojin, et al.
Publicado: (2025)

Query-Aware Learnable Graph Pooling Tokens as Prompt for Large Language Models
por: Kim, Wooyoung, et al.
Publicado: (2025)

Parallel Tokenizers: Rethinking Vocabulary Design for Cross-Lingual Transfer
por: Kautsar, Muhammad Dehan Al, et al.
Publicado: (2025)

LAST: Language Model Aware Speech Tokenization
por: Turetzky, Arnon, et al.
Publicado: (2024)

EGAD: Entropy-Guided Adaptive Distillation for Token-Level Knowledge Transfer
por: Zhang, Hao, et al.
Publicado: (2026)

Token-Level Uncertainty-Aware Objective for Language Model Post-Training
por: Liu, Tingkai, et al.
Publicado: (2025)

LM-PUB-QUIZ: A Comprehensive Framework for Zero-Shot Evaluation of Relational Knowledge in Language Models
por: Ploner, Max, et al.
Publicado: (2024)

TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees
por: Liu, Tianyu, et al.
Publicado: (2026)

Every Little Helps: Building Knowledge Graph Foundation Model with Fine-grained Transferable Multi-modal Tokens
por: Zhang, Yichi, et al.
Publicado: (2026)

Problematic Tokens: Tokenizer Bias in Large Language Models
por: Yang, Jin, et al.
Publicado: (2024)

Can Models Help Us Create Better Models? Evaluating LLMs as Data Scientists
por: Pietruszka, Michał, et al.
Publicado: (2024)

Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine
por: Akhynko, Kateryna, et al.
Publicado: (2025)

An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation
por: Trokhymovych, Mykola, et al.
Publicado: (2026)

Towards Linguistically-Aware and Language-Independent Tokenization for Large Language Models (LLMs)
por: Rahman, Abrar, et al.
Publicado: (2024)

Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
por: Yang, Jinbiao
Publicado: (2024)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
por: Zeng, Runjia, et al.
Publicado: (2026)

Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
por: Wang, Junlin, et al.
Publicado: (2024)

Towards a Principled Evaluation of Knowledge Editors
por: Pohl, Sebastian, et al.
Publicado: (2025)

ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection
por: Herrlein, Janek, et al.
Publicado: (2024)

Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need
por: Skiredj, Abderrahman, et al.
Publicado: (2024)

AnTKV: Anchor Token-Aware Sub-Bit Vector Quantization for KV Cache in Large Language Models
por: Li, Zeyu, et al.
Publicado: (2025)

Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning
por: Ye, Ziang, et al.
Publicado: (2024)

Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer
por: Lee, Seungyoon, et al.
Publicado: (2025)

SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling
por: Zhao, Anhao, et al.
Publicado: (2025)

DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens
por: Chen, Shaoshen, et al.
Publicado: (2025)

Positional Encoding via Token-Aware Phase Attention
por: Wang, Yu, et al.
Publicado: (2025)

The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Tasks
por: Ebing, Benedikt, et al.
Publicado: (2025)

Training-Trajectory-Aware Token Selection
por: Shen, Zhanming, et al.
Publicado: (2026)

Token-Budget-Aware LLM Reasoning
por: Han, Tingxu, et al.
Publicado: (2024)

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training
por: Qing, Liu, et al.
Publicado: (2026)