Cover Image

MorphTok: Morphologically Grounded Tokenization for Indian Languages

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Brahma, Maharaj, Karthika, N J, Singh, Atul, Adiga, Devaraj, Bhate, Smruti, Ramakrishnan, Ganesh, Saluja, Rohit, Desarkar, Maunendra Sankar
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multilingual Tokenization through the Lens of Indian Languages: Challenges and Insights
by: Karthika, N J, et al.
Published: (2025)

DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Context
by: Sahoo, Pramit, et al.
Published: (2025)

NLIP_Lab-IITH Low-Resource MT System for WMT24 Indic MT Shared Task
by: Sahoo, Pramit, et al.
Published: (2024)

NLIP_Lab-IITH Multilingual MT System for WAT24 MT Shared Task
by: Brahma, Maharaj, et al.
Published: (2024)

The Art of Breaking Words: Rethinking Multilingual Tokenizer Design
by: Thakur, Aamod, et al.
Published: (2025)

BoK: Introducing Bag-of-Keywords Loss for Interpretable Dialogue Response Generation
by: Dey, Suvodip, et al.
Published: (2025)

AyurParam: A State-of-the-Art Bilingual Language Model for Ayurveda
by: Nauman, Mohd, et al.
Published: (2025)

Unsupervised Domain Adaptation with Global and Local Graph Neural Networks in Limited Labeled Data Scenario: Application to Disaster Management
by: Ghosh, Samujjwal, et al.
Published: (2021)

Transformer based Multitask Learning for Image Captioning and Object Detection
by: Basak, Debolena, et al.
Published: (2024)

CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages
by: Maurya, Kaushal Kumar, et al.
Published: (2023)

LEVOS: Leveraging Vocabulary Overlap with Sanskrit to Generate Technical Lexicons in Indian Languages
by: J, Karthika N, et al.
Published: (2024)

Semantically Cohesive Word Grouping in Indian Languages
by: Karthika, N J, et al.
Published: (2025)

A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs
by: Singh, Vaibhav, et al.
Published: (2024)

LexGen: Domain-aware Multilingual Lexicon Generation
by: Maheshwari, Ayush, et al.
Published: (2024)

Improving Video Question Answering through query-based frame selection
by: Patil, Himanshu, et al.
Published: (2026)

Intent Aware Context Retrieval for Multi-Turn Agricultural Question Answering
by: Vijayvargia, Abhay, et al.
Published: (2025)

A2TTS: TTS for Low Resource Indian Languages
by: Bhadoriya, Ayush Singh, et al.
Published: (2025)

Technical report: Impact of Duration Prediction on Speaker-specific TTS for Indian Languages
by: Pandey, Isha, et al.
Published: (2025)

INDIC DIALECT: A Multi Task Benchmark to Evaluate and Translate in Indian Language Dialects
by: Sharma, Tarun, et al.
Published: (2026)

Samasāmayik: A Parallel Dataset for Hindi-Sanskrit Machine Translation
by: Karthika, N J, et al.
Published: (2026)

DRISHTIKON: Visual Grounding at Multiple Granularities in Documents
by: Kasuba, Badri Vishal, et al.
Published: (2025)

Assessing the Sustainability and Food Security Implications of Introduced Pangasianodon hypophthalmus (Sauvage, 1878) and Clarias gariepinus (Burchell, 1822) Into Indian Aquaculture: A OneHealth Perspective
by: Atul K. Singh
Published: (2024)

VoiceMorph: How AI Voice Morphing Reveals the Boundaries of Auditory Self-Recognition
by: Shimizu, Kye, et al.
Published: (2025)

MorphPiece : A Linguistic Tokenizer for Large Language Models
by: Jabbar, Haris
Published: (2023)

Vacuum‐Assisted Microwave Pyrolysis of Rice Husk for the Production of High‐Heating Value Biochar for Practical Energy Applications
by: Maninderjeet Singh, et al.
Published: (2025)

Management of Biological Diversity of Indian Forests
by: Sathe, Atul
Published: (2011)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
by: Cho, Gyeongje, et al.
Published: (2025)

BrainMorph: A Foundational Keypoint Model for Robust and Flexible Brain MRI Registration
by: Wang, Alan Q., et al.
Published: (2024)

TokTalk: Expressive Real-time Facial Animation from Audio-LLM Tokens
by: Zhao, Qingcheng, et al.
Published: (2026)

SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
by: Tănase, Andrei-Valentin, et al.
Published: (2025)

PyraTok: Language-Aligned Pyramidal Tokenizer for Video Understanding and Generation
by: Susladkar, Onkar, et al.
Published: (2026)

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
by: Altıntaş, Gül Sena, et al.
Published: (2025)

TokMem: One-Token Procedural Memory for Large Language Models
by: Wu, Zijun, et al.
Published: (2025)

Lang2Morph: Language-Driven Morphological Design of Robotic Hands
by: Qiao, Yanyuan, et al.
Published: (2025)

RoboMorph: Evolving Robot Morphology using Large Language Models
by: Qiu, Kevin, et al.
Published: (2024)

Dementia Strategies of England and Glasgow Declaration: What Requires to be Done?
by: Smruti Bulsari
Published: (2024)

Sāmayik: A Benchmark and Dataset for English-Sanskrit Translation
by: Maheshwari, Ayush, et al.
Published: (2023)

Beyond Rice: The New Dynamics of Food, Fuel, and Farming in India
by: Pooja Saluja
Published: (2025)

MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies
by: Asgari, Ehsaneddin, et al.
Published: (2025)

Realtime Dynamic Gaze Target Tracking and Depth-Level Estimation
by: Seraj, Esmaeil, et al.
Published: (2024)