TurkBench: A Benchmark for Evaluating Turkish Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917245140074496 |
|---|---|
| author | Toraman, Çağrı Sever, Ahmet Kaan Cengiz, Ayse Aysu Arslan, Elif Ecem Sevinç, Görkem Birdal, Mete Mert Güldemir, Yusuf Faruk Kanburoğlu, Ali Buğra Felekoğlu, Sezen Gürlek, Osman Kantar, Sarp Kütük, Birsen Şahin Tufan, Büşra Genç, Elif Coşkun, Serkan Demir, Gupse Ekin Arayıcı, Muhammed Emin Dursun, Olgun Gungor, Onur Üsküdarlı, Susan Topraksoy, Abdullah Darıcı, Esra |
| author_facet | Toraman, Çağrı Sever, Ahmet Kaan Cengiz, Ayse Aysu Arslan, Elif Ecem Sevinç, Görkem Birdal, Mete Mert Güldemir, Yusuf Faruk Kanburoğlu, Ali Buğra Felekoğlu, Sezen Gürlek, Osman Kantar, Sarp Kütük, Birsen Şahin Tufan, Büşra Genç, Elif Coşkun, Serkan Demir, Gupse Ekin Arayıcı, Muhammed Emin Dursun, Olgun Gungor, Onur Üsküdarlı, Susan Topraksoy, Abdullah Darıcı, Esra |
| contents | With the recent surge in the development of large language models, the need for comprehensive and language-specific evaluation benchmarks has become critical. While significant progress has been made in evaluating English-language models, benchmarks for other languages, particularly those with unique linguistic characteristics such as Turkish, remain less developed. Our study introduces TurkBench, a comprehensive benchmark designed to assess the capabilities of generative large language models in the Turkish language. TurkBench involves 8,151 data samples across 21 distinct subtasks. These are organized under six main categories of evaluation: Knowledge, Language Understanding, Reasoning, Content Moderation, Turkish Grammar and Vocabulary, and Instruction Following. The diverse range of tasks and the culturally relevant data would provide researchers and developers with a valuable tool for evaluating their models and identifying areas for improvement. We further publish our benchmark for online submissions at https://huggingface.co/turkbench |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_07020 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TurkBench: A Benchmark for Evaluating Turkish Large Language Models Toraman, Çağrı Sever, Ahmet Kaan Cengiz, Ayse Aysu Arslan, Elif Ecem Sevinç, Görkem Birdal, Mete Mert Güldemir, Yusuf Faruk Kanburoğlu, Ali Buğra Felekoğlu, Sezen Gürlek, Osman Kantar, Sarp Kütük, Birsen Şahin Tufan, Büşra Genç, Elif Coşkun, Serkan Demir, Gupse Ekin Arayıcı, Muhammed Emin Dursun, Olgun Gungor, Onur Üsküdarlı, Susan Topraksoy, Abdullah Darıcı, Esra Computation and Language Artificial Intelligence With the recent surge in the development of large language models, the need for comprehensive and language-specific evaluation benchmarks has become critical. While significant progress has been made in evaluating English-language models, benchmarks for other languages, particularly those with unique linguistic characteristics such as Turkish, remain less developed. Our study introduces TurkBench, a comprehensive benchmark designed to assess the capabilities of generative large language models in the Turkish language. TurkBench involves 8,151 data samples across 21 distinct subtasks. These are organized under six main categories of evaluation: Knowledge, Language Understanding, Reasoning, Content Moderation, Turkish Grammar and Vocabulary, and Instruction Following. The diverse range of tasks and the culturally relevant data would provide researchers and developers with a valuable tool for evaluating their models and identifying areas for improvement. We further publish our benchmark for online submissions at https://huggingface.co/turkbench |
| title | TurkBench: A Benchmark for Evaluating Turkish Large Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.07020 |