Understanding and Mitigating Tokenization Bias in Language Models

Documenti analoghi

• Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles
  di: Phan, Buu, et al.
  Pubblicazione: (2024)
• Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
  di: Phan, Buu, et al.
  Pubblicazione: (2025)
• Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models
  di: Adiga, Rishabh, et al.
  Pubblicazione: (2024)
• TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
  di: Choo, Jinho, et al.
  Pubblicazione: (2026)
• Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
  di: Zhang, Zhongjian, et al.
  Pubblicazione: (2026)