Image de couverture de livre

Building pre-train LLM Dataset for the INDIC Languages: a case study on Hindi

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Parida, Shantipriya, Panwar, Shakshi, Lata, Kusum, Mishra, Sanskruti, Sekhar, Sambit
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Towards a More Inclusive AI: Progress and Perspectives in Large Language Model Training for the Sámi Language
par: Paul, Ronny, et autres
Publié: (2024)

HindiLLM: Large Language Model for Hindi
par: Chouhan, Sanjay, et autres
Publié: (2024)

Airavata: Introducing Hindi Instruction-tuned LLM
par: Gala, Jay, et autres
Publié: (2024)

Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation
par: Singh, Sumit, et autres
Publié: (2025)

DeepRAG: Building a Custom Hindi Embedding Model for Retrieval Augmented Generation from Scratch
par: M, Nandakishor
Publié: (2025)

Bridging the Data Gap: Creating a Hindi Text Summarization Dataset from the English XSUM
par: Katwe, Praveenkumar, et autres
Publié: (2026)

Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi
par: Fatimah, Shiza, et autres
Publié: (2026)

Suvach -- Generated Hindi QA benchmark
par: Narayanan, Vaishak, et autres
Publié: (2024)

COMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing
par: Sheth, Rajvee, et autres
Publié: (2025)

Multi-class Regret Detection in Hindi Devanagari Script
par: Sharma, Renuka, et autres
Publié: (2024)

Survey of Pseudonymization, Abstractive Summarization & Spell Checker for Hindi and Marathi
par: Ransing, Rasika, et autres
Publié: (2024)

In-domain SSL pre-training and streaming ASR
par: Duret, Jarod, et autres
Publié: (2025)

HLDC: Hindi Legal Documents Corpus
par: Kapoor, Arnav, et autres
Publié: (2022)

Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English
par: Anand, Avinash, et autres
Publié: (2024)

Multilingual LLMs Are Not Multilingual Thinkers: Evidence from Hindi Analogy Evaluation
par: Gupta, Ashray, et autres
Publié: (2025)

CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models
par: Pistilli, Giada, et autres
Publié: (2024)

'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMs
par: Joshi, Ishika, et autres
Publié: (2024)

Building a Multivariate Time Series Benchmarking Datasets Inspired by Natural Language Processing (NLP)
par: Mustafa, Mohammad Asif Ibna, et autres
Publié: (2024)

LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models
par: Veldanda, Akshaj Kumar, et autres
Publié: (2024)

Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar
par: Htet, Aung Kyaw, et autres
Publié: (2025)

An Under-Explored Application for Explainable Multimodal Misogyny Detection in code-mixed Hindi-English
par: Yadav, Sargam, et autres
Publié: (2026)

Explainable cognitive decline detection in free dialogues with a Machine Learning approach based on pre-trained Large Language Models
par: de Arriba-Pérez, Francisco, et autres
Publié: (2024)

LLM Compression: How Far Can We Go in Balancing Size and Performance?
par: Sk, Sahil, et autres
Publié: (2025)

CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation
par: Zhao, Jingqian, et autres
Publié: (2025)

MedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queries
par: Ghosh, Akash, et autres
Publié: (2024)

Can pre-trained language models generate titles for research papers?
par: Rehman, Tohida, et autres
Publié: (2024)

On the Suitability of pre-trained foundational LLMs for Analysis in German Legal Education
par: Wendlinger, Lorenz, et autres
Publié: (2024)

T1: A Tool-Oriented Conversational Dataset for Multi-Turn Agentic Planning
par: Chakraborty, Amartya, et autres
Publié: (2025)

Building a Rich Dataset to Empower the Persian Question Answering Systems
par: Yazdinejad, Mohsen, et autres
Publié: (2024)

The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories
par: Shah, Raj Sanjay, et autres
Publié: (2025)

Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
par: Ye, Liqin, et autres
Publié: (2025)

Planning in the LLM Era: Building for Reliability and Efficiency
par: Katz, Michael, et autres
Publié: (2026)

Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales
par: Reuben, Maor, et autres
Publié: (2024)

WizardLM: Empowering large pre-trained language models to follow complex instructions
par: Xu, Can, et autres
Publié: (2023)

Scaling LLM Pre-training with Vocabulary Curriculum
par: Yu, Fangyuan
Publié: (2025)

Revealing the impact of synthetic native samples and multi-tasking strategies in Hindi-English code-mixed humour and sarcasm detection
par: Mazumder, Debajyoti, et autres
Publié: (2024)

Human-1 by Josh Talks: A Full-Duplex Conversational Modeling Framework in Hindi using Real-World Conversations
par: Singh, Bhaskar, et autres
Publié: (2026)

Building Models of Neurological Language
par: Watkins, Henry
Publié: (2025)

No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models
par: Kumar, Charaka Vinayak, et autres
Publié: (2025)

Advantages of Domain Knowledge Injection for Legal Document Summarization: A Case Study on Summarizing Indian Court Judgments in English and Hindi
par: Datta, Debtanu, et autres
Publié: (2026)