Copertina

Comparative Analysis of Tokenization Algorithms for Low-Resource Language Dzongkha

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wangchuk, Tandin, Gonsalves, Tad
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Multilingual Question Answering in Low-Resource Settings: A Dzongkha-English Benchmark for Foundation Models
di: Hosain, Md. Tanzib, et al.
Pubblicazione: (2025)

Biophysical characteristics, distribution and conservation challenges of critically endangered Bhutan poppy (Papaver bhutanicum) in Jigme Dorji National Park, Bhutan
di: Pema Dendup, et al.
Pubblicazione: (2024)

Reducing Tokenization Premiums for Low-Resource Languages
di: Churchill, Geoffrey, et al.
Pubblicazione: (2026)

Zero-shot OCR Accuracy of Low-Resourced Languages: A Comparative Analysis on Sinhala and Tamil
di: Jayatilleke, Nevidu, et al.
Pubblicazione: (2025)

Tokenization Strategies for Low-Resource Agglutinative Languages in Word2Vec: Case Study on Turkish and Finnish
di: Hu, Jinfan Frank
Pubblicazione: (2025)

Stop Jostling: Adaptive Negative Sampling Reduces the Marginalization of Low-Resource Language Tokens by Cross-Entropy Loss
di: Turumtaev, Galim
Pubblicazione: (2026)

Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts
di: Shen, Yanxin, et al.
Pubblicazione: (2024)

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models
di: Raj, Bharath, et al.
Pubblicazione: (2024)

Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLP
di: Remy, François, et al.
Pubblicazione: (2024)

Comparative Analysis of Different Efficient Fine Tuning Methods of Large Language Models (LLMs) in Low-Resource Setting
di: Srinivasan, Krishna Prasad Varadarajan, et al.
Pubblicazione: (2024)

Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
di: Ganescu, Bianca-Mihaela, et al.
Pubblicazione: (2025)

The Zeno's Paradox of `Low-Resource' Languages
di: Nigatu, Hellina Hailu, et al.
Pubblicazione: (2024)

Bootstrapping Embeddings for Low Resource Languages
di: Basoz, Merve, et al.
Pubblicazione: (2026)

GlotLID: Language Identification for Low-Resource Languages
di: Kargaran, Amir Hossein, et al.
Pubblicazione: (2023)

Algorithm for Semantic Network Generation from Texts of Low Resource Languages Such as Kiswahili
di: Wanjawa, Barack Wamkaya, et al.
Pubblicazione: (2025)

Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
di: Abonizio, Hugo, et al.
Pubblicazione: (2025)

Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs
di: Wang, Dingdong, et al.
Pubblicazione: (2025)

Explaining Pre-Trained Language Models with Attribution Scores: An Analysis in Low-Resource Settings
di: Zhou, Wei, et al.
Pubblicazione: (2024)

Adapting Large Language Models to a Low-Resource Agglutinative Language: A Comparative Study of LoRA and QLoRA for Bashkir
di: Arabov, Mullosharaf K., et al.
Pubblicazione: (2026)

Investigating Hallucination in Conversations for Low Resource Languages
di: Das, Amit, et al.
Pubblicazione: (2025)

On Multilingual Encoder Language Model Compression for Low-Resource Languages
di: Gurgurov, Daniil, et al.
Pubblicazione: (2025)

TriLex: A Framework for Multilingual Sentiment Analysis in Low-Resource South African Languages
di: Nkongolo, Mike, et al.
Pubblicazione: (2025)

Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon
di: Koto, Fajri, et al.
Pubblicazione: (2024)

Protection beyond park boundaries: On the impact of systematic patrolling in Divisional Forest Office, Sarpang, Bhutan
di: Phub Dhendup, et al.
Pubblicazione: (2024)

Large Language Models for Toxic Language Detection in Low-Resource Balkan Languages
di: Muminovic, Amel, et al.
Pubblicazione: (2025)

LMSpell: Neural Spell Checking for Low-Resource Languages
di: Gunathilake, Akesh, et al.
Pubblicazione: (2025)

Massively Multilingual Text Translation For Low-Resource Languages
di: Zhou, Zhong
Pubblicazione: (2024)

LLM Probe: Evaluating LLMs for Low-Resource Languages
di: Teklehaymanot, Hailay Kidu, et al.
Pubblicazione: (2026)

LLMs for Extremely Low-Resource Finno-Ugric Languages
di: Purason, Taido, et al.
Pubblicazione: (2024)

Task Arithmetic with Support Languages for Low-Resource ASR
di: Rafkin, Emma, et al.
Pubblicazione: (2026)

Unsupervised Bilingual Lexicon Induction for Low Resource Languages
di: Rathnayake, Charitha, et al.
Pubblicazione: (2024)

Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5
di: Dang, Thao Anh, et al.
Pubblicazione: (2024)

Performance of Recent Large Language Models for a Low-Resourced Language
di: Jayakody, Ravindu, et al.
Pubblicazione: (2024)

Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
di: Phan, Buu, et al.
Pubblicazione: (2025)

Introducing Syllable Tokenization for Low-resource Languages: A Case Study with Swahili
di: Atuhurra, Jesse, et al.
Pubblicazione: (2024)

Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource Languages
di: Puranegedara, Imalsha, et al.
Pubblicazione: (2025)

Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?
di: Song, Yewei, et al.
Pubblicazione: (2025)

Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs
di: Zhou, Huichi, et al.
Pubblicazione: (2025)

Evaluating and Adapting Large Language Models to Represent Folktales in Low-Resource Languages
di: Meaney, JA, et al.
Pubblicazione: (2024)

Towards Neural No-Resource Language Translation: A Comparative Evaluation of Approaches
di: Thakur, Madhavendra
Pubblicazione: (2024)