Imagen de Portada

Krutrim LLM: A Novel Tokenization Strategy for Multilingual Indic Languages with Petabyte-Scale Data Processing

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kumar, Rahul, Kakde, Shubham, Rajput, Divyansh, Ibrahim, Daud, Nahata, Rishabh, Sowjanya, Pidathala, Kumarr, Deepak, Bhargava, Gautam, Khatri, Chandra
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Krutrim LLM: Multilingual Foundational Model for over a Billion People
por: Kallappa, Aditya, et al.
Publicado: (2025)

IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
por: Faraz, Ali, et al.
Publicado: (2025)

MUTANT: A Recipe for Multilingual Tokenizer Design
por: Rana, Souvik, et al.
Publicado: (2025)

BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
por: Manoj, Guduru, et al.
Publicado: (2025)

TIRE X Theory V3 (Temporal Inertial Reversal & Expansion – X)
por: Chandra, Divyansh
Publicado: (2025)

TIRE X Theory V2 (Temporal Inertial Reversal & Expansion – X)
por: Chandra, Divyansh
Publicado: (2025)

Multilingual State Space Models for Structured Question Answering in Indic Languages
por: Vats, Arpita, et al.
Publicado: (2025)

Culture Roots Vs. Modernity in Mahesh Elkunchwar’s Old Stone Mansion
por: Haresh Kakde
Publicado: (2017)

VoiceAgentBench: Are Voice Assistants ready for agentic tasks?
por: Jain, Dhruv, et al.
Publicado: (2025)

Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
por: Rachamalla, Neel Prabhanjan, et al.
Publicado: (2025)

IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languages
por: Endait, Sharvi, et al.
Publicado: (2025)

IndicMedDialog: A Parallel Multi-Turn Medical Dialogue Dataset for Accessible Healthcare in Indic Languages
por: Nigam, Shubham Kumar, et al.
Publicado: (2026)

IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
por: Singh, Harman, et al.
Publicado: (2024)

Sacred Architecture and Energy Flow: A Scientific Study of Vastu Principles in Built Environments
por: Acharya Pt Dr Avdhesh Kumarr
Publicado: (2026)

Camphor & Consciousness: Ancient Wisdom Meets Modern Science
por: Acharya Pt Dr Avdhesh Kumarr
Publicado: (2026)

Reincarnation in Hindu Dharma: Death as Transformation of Energy and Continuity of Spiritual Missions (1600–2000)
por: Acharya Pt Dr Avdhesh Kumarr
Publicado: (2026)

Data Caching for Enterprise-Grade Petabyte-Scale OLAP
por: Tang, Chunxu, et al.
Publicado: (2024)

Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages
por: Khan, Shaharukh, et al.
Publicado: (2026)

Knowledge-enhanced Transformer for Multivariate Long Sequence Time-series Forecasting
por: Kakde, Shubham Tanaji, et al.
Publicado: (2024)

Ranks of Matrices of Logarithms of Algebraic Numbers II: The Matrix Coefficient Conjecture
por: Dasgupta, Samit, et al.
Publicado: (2024)

IndicSentEval: How Effectively do Multilingual Transformer Models encode Linguistic Properties for Indic Languages?
por: Aravapalli, Akhilesh, et al.
Publicado: (2024)

Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
por: Panchal, Mihir, et al.
Publicado: (2026)

Improving Multilingual Neural Machine Translation System for Indic Languages
por: Das, Sudhansu Bala, et al.
Publicado: (2022)

Molecular and Cellular Mechanisms of Sarcopenia: Integrating Fiber‐Type Remodeling, Contractile Protein Dynamics, and Systemic Regulatory Pathways
por: Nikam Rutuja Dilesh, et al.
Publicado: (2026)

Microservice Architecture Patterns for Scalable Machine Learning Systems
por: Karanam, Sowjanya, et al.
Publicado: (2026)

Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR
por: Mazumder, Debajyoti, et al.
Publicado: (2026)

Tokenization Matters: Improving Zero-Shot NER for Indic Languages
por: Pattnayak, Priyaranjan, et al.
Publicado: (2025)

IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages
por: Dawar, Aviral, et al.
Publicado: (2026)

Perovskite Materials for Efficient Utilization of Thermal Energy
por: Vishwajit M. Gaikwad, et al.
Publicado: (2025)

Equivalence between tying and vertical integration with Pareto‐improving foreclosure
por: Yong Chao, et al.
Publicado: (2026)

Exploring Damage and Penetration in Soft Armors Under Ballistic Impact Through a Novel and Efficient 3D Peridynamic Model
por: Daud Ali Abdoh
Publicado: (2025)

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia
por: Pattnayak, Priyaranjan, et al.
Publicado: (2026)

IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages
por: Javed, Tahir, et al.
Publicado: (2024)

First-principles investigation of thermodynamics and electronic transitions in vacancy-ordered rare-earth perovskite nickelates
por: Bhagat, Devang, et al.
Publicado: (2024)

Parallel Corpora for Machine Translation in Low-resource Indic Languages: A Comprehensive Review
por: Raja, Rahul, et al.
Publicado: (2025)

A Hypergraph based lower bound on Pliable Index Coding based on Nested Side-Information Sets
por: B., Tulasi Sowjanya, et al.
Publicado: (2025)

An Improved ECC‐Based Authenticated Key Exchange Protocol for Industrial IoT Environments
por: K. Somasena Reddy, et al.
Publicado: (2026)

WaAOA_DenseNet: Walrus Archimedes Optimization Algorithm‐Based Trust Updation in UAV‐Assisted Wireless Sensor Network
por: Sowjanya Nagulapati, et al.
Publicado: (2025)

A Brief Review on Microcellulose Hydrogel
por: Vaibhav Kakde*, Priyanshu Thakre, Praful Hire
Publicado: (2025)

On a refinement of the Birch and Swinnerton-Dyer Conjecture in positive characteristic
por: Burns, David, et al.
Publicado: (2018)