Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.06011 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917980825190400 |
|---|---|
| author | Choudhury, Monojit Chauhan, Shivam Das, Rocktim Jyoti Sahnan, Dhruv Han, Xudong Li, Haonan Singh, Aaryamonvikram Jadhav, Alok Anil Agarwal, Utkarsh Choudhary, Mukund Banerjee, Debopriyo Koto, Fajri Bhat, Junaid Shukla, Awantika Ghosh, Samujjwal Kamboj, Samta Pandit, Onkar Pradhan, Lalit Pal, Rahul Sahu, Sunil Doraiswamy, Soundar Mullah, Parvez Filali, Ali El Sengupta, Neha Ramakrishnan, Gokul Joshi, Rituraj Gosal, Gurpreet Sheinin, Avraham Vassilieva, Natalia Nakov, Preslav |
| author_facet | Choudhury, Monojit Chauhan, Shivam Das, Rocktim Jyoti Sahnan, Dhruv Han, Xudong Li, Haonan Singh, Aaryamonvikram Jadhav, Alok Anil Agarwal, Utkarsh Choudhary, Mukund Banerjee, Debopriyo Koto, Fajri Bhat, Junaid Shukla, Awantika Ghosh, Samujjwal Kamboj, Samta Pandit, Onkar Pradhan, Lalit Pal, Rahul Sahu, Sunil Doraiswamy, Soundar Mullah, Parvez Filali, Ali El Sengupta, Neha Ramakrishnan, Gokul Joshi, Rituraj Gosal, Gurpreet Sheinin, Avraham Vassilieva, Natalia Nakov, Preslav |
| contents | Developing high-quality large language models (LLMs) for moderately resourced languages presents unique challenges in data availability, model adaptation, and evaluation. We introduce Llama-3-Nanda-10B-Chat, or Nanda for short, a state-of-the-art Hindi-centric instruction-tuned generative LLM, designed to push the boundaries of open-source Hindi language models. Built upon Llama-3-8B, Nanda incorporates continuous pre-training with expanded transformer blocks, leveraging the Llama Pro methodology. A key challenge was the limited availability of high-quality Hindi text data; we addressed this through rigorous data curation, augmentation, and strategic bilingual training, balancing Hindi and English corpora to optimize cross-linguistic knowledge transfer. With 10 billion parameters, Nanda stands among the top-performing open-source Hindi and multilingual models of similar scale, demonstrating significant advantages over many existing models. We provide an in-depth discussion of training strategies, fine-tuning techniques, safety alignment, and evaluation metrics, demonstrating how these approaches enabled Nanda to achieve state-of-the-art results. By open-sourcing Nanda, we aim to advance research in Hindi LLMs and support a wide range of real-world applications across academia, industry, and public services. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06011 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi Choudhury, Monojit Chauhan, Shivam Das, Rocktim Jyoti Sahnan, Dhruv Han, Xudong Li, Haonan Singh, Aaryamonvikram Jadhav, Alok Anil Agarwal, Utkarsh Choudhary, Mukund Banerjee, Debopriyo Koto, Fajri Bhat, Junaid Shukla, Awantika Ghosh, Samujjwal Kamboj, Samta Pandit, Onkar Pradhan, Lalit Pal, Rahul Sahu, Sunil Doraiswamy, Soundar Mullah, Parvez Filali, Ali El Sengupta, Neha Ramakrishnan, Gokul Joshi, Rituraj Gosal, Gurpreet Sheinin, Avraham Vassilieva, Natalia Nakov, Preslav Computation and Language Developing high-quality large language models (LLMs) for moderately resourced languages presents unique challenges in data availability, model adaptation, and evaluation. We introduce Llama-3-Nanda-10B-Chat, or Nanda for short, a state-of-the-art Hindi-centric instruction-tuned generative LLM, designed to push the boundaries of open-source Hindi language models. Built upon Llama-3-8B, Nanda incorporates continuous pre-training with expanded transformer blocks, leveraging the Llama Pro methodology. A key challenge was the limited availability of high-quality Hindi text data; we addressed this through rigorous data curation, augmentation, and strategic bilingual training, balancing Hindi and English corpora to optimize cross-linguistic knowledge transfer. With 10 billion parameters, Nanda stands among the top-performing open-source Hindi and multilingual models of similar scale, demonstrating significant advantages over many existing models. We provide an in-depth discussion of training strategies, fine-tuning techniques, safety alignment, and evaluation metrics, demonstrating how these approaches enabled Nanda to achieve state-of-the-art results. By open-sourcing Nanda, we aim to advance research in Hindi LLMs and support a wide range of real-world applications across academia, industry, and public services. |
| title | Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2504.06011 |