Image de couverture de livre

MorphTok: Morphologically Grounded Tokenization for Indian Languages

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Brahma, Maharaj, Karthika, N J, Singh, Atul, Adiga, Devaraj, Bhate, Smruti, Ramakrishnan, Ganesh, Saluja, Rohit, Desarkar, Maunendra Sankar
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Multilingual Tokenization through the Lens of Indian Languages: Challenges and Insights
par: Karthika, N J, et autres
Publié: (2025)

DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Context
par: Sahoo, Pramit, et autres
Publié: (2025)

NLIP_Lab-IITH Low-Resource MT System for WMT24 Indic MT Shared Task
par: Sahoo, Pramit, et autres
Publié: (2024)

NLIP_Lab-IITH Multilingual MT System for WAT24 MT Shared Task
par: Brahma, Maharaj, et autres
Publié: (2024)

The Art of Breaking Words: Rethinking Multilingual Tokenizer Design
par: Thakur, Aamod, et autres
Publié: (2025)

BoK: Introducing Bag-of-Keywords Loss for Interpretable Dialogue Response Generation
par: Dey, Suvodip, et autres
Publié: (2025)

AyurParam: A State-of-the-Art Bilingual Language Model for Ayurveda
par: Nauman, Mohd, et autres
Publié: (2025)

Unsupervised Domain Adaptation with Global and Local Graph Neural Networks in Limited Labeled Data Scenario: Application to Disaster Management
par: Ghosh, Samujjwal, et autres
Publié: (2021)

Transformer based Multitask Learning for Image Captioning and Object Detection
par: Basak, Debolena, et autres
Publié: (2024)

CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages
par: Maurya, Kaushal Kumar, et autres
Publié: (2023)

LEVOS: Leveraging Vocabulary Overlap with Sanskrit to Generate Technical Lexicons in Indian Languages
par: J, Karthika N, et autres
Publié: (2024)

Semantically Cohesive Word Grouping in Indian Languages
par: Karthika, N J, et autres
Publié: (2025)

A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs
par: Singh, Vaibhav, et autres
Publié: (2024)

LexGen: Domain-aware Multilingual Lexicon Generation
par: Maheshwari, Ayush, et autres
Publié: (2024)

Improving Video Question Answering through query-based frame selection
par: Patil, Himanshu, et autres
Publié: (2026)

Intent Aware Context Retrieval for Multi-Turn Agricultural Question Answering
par: Vijayvargia, Abhay, et autres
Publié: (2025)

A2TTS: TTS for Low Resource Indian Languages
par: Bhadoriya, Ayush Singh, et autres
Publié: (2025)

Technical report: Impact of Duration Prediction on Speaker-specific TTS for Indian Languages
par: Pandey, Isha, et autres
Publié: (2025)

INDIC DIALECT: A Multi Task Benchmark to Evaluate and Translate in Indian Language Dialects
par: Sharma, Tarun, et autres
Publié: (2026)

Samasāmayik: A Parallel Dataset for Hindi-Sanskrit Machine Translation
par: Karthika, N J, et autres
Publié: (2026)

DRISHTIKON: Visual Grounding at Multiple Granularities in Documents
par: Kasuba, Badri Vishal, et autres
Publié: (2025)

Assessing the Sustainability and Food Security Implications of Introduced Pangasianodon hypophthalmus (Sauvage, 1878) and Clarias gariepinus (Burchell, 1822) Into Indian Aquaculture: A OneHealth Perspective
par: Atul K. Singh
Publié: (2024)

VoiceMorph: How AI Voice Morphing Reveals the Boundaries of Auditory Self-Recognition
par: Shimizu, Kye, et autres
Publié: (2025)

MorphPiece : A Linguistic Tokenizer for Large Language Models
par: Jabbar, Haris
Publié: (2023)

Vacuum‐Assisted Microwave Pyrolysis of Rice Husk for the Production of High‐Heating Value Biochar for Practical Energy Applications
par: Maninderjeet Singh, et autres
Publié: (2025)

Management of Biological Diversity of Indian Forests
par: Sathe, Atul
Publié: (2011)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
par: Cho, Gyeongje, et autres
Publié: (2025)

BrainMorph: A Foundational Keypoint Model for Robust and Flexible Brain MRI Registration
par: Wang, Alan Q., et autres
Publié: (2024)

TokTalk: Expressive Real-time Facial Animation from Audio-LLM Tokens
par: Zhao, Qingcheng, et autres
Publié: (2026)

SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
par: Tănase, Andrei-Valentin, et autres
Publié: (2025)

PyraTok: Language-Aligned Pyramidal Tokenizer for Video Understanding and Generation
par: Susladkar, Onkar, et autres
Publié: (2026)

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
par: Altıntaş, Gül Sena, et autres
Publié: (2025)

TokMem: One-Token Procedural Memory for Large Language Models
par: Wu, Zijun, et autres
Publié: (2025)

Lang2Morph: Language-Driven Morphological Design of Robotic Hands
par: Qiao, Yanyuan, et autres
Publié: (2025)

RoboMorph: Evolving Robot Morphology using Large Language Models
par: Qiu, Kevin, et autres
Publié: (2024)

Dementia Strategies of England and Glasgow Declaration: What Requires to be Done?
par: Smruti Bulsari
Publié: (2024)

Beyond Rice: The New Dynamics of Food, Fuel, and Farming in India
par: Pooja Saluja
Publié: (2025)

Sāmayik: A Benchmark and Dataset for English-Sanskrit Translation
par: Maheshwari, Ayush, et autres
Publié: (2023)

MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies
par: Asgari, Ehsaneddin, et autres
Publié: (2025)

DINO-Tok: Adapting DINO for Visual Tokenizers
par: Jia, Mingkai, et autres
Publié: (2025)