Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2606.02404 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913180290121728 |
|---|---|
| author | Lee, Nahyun Yoon, Dongkeun Son, Guijin Kim, Geewook Ko, Dayoon Park, Jeonghun Yoo, Haneul Cho, Jaewon Park, Junghun Lee, Changyoon Jang, Kyochul Kim, Jaeyeon Kim, Eunsu Cho, Woojin Kim, Seungone |
| author_facet | Lee, Nahyun Yoon, Dongkeun Son, Guijin Kim, Geewook Ko, Dayoon Park, Jeonghun Yoo, Haneul Cho, Jaewon Park, Junghun Lee, Changyoon Jang, Kyochul Kim, Jaeyeon Kim, Eunsu Cho, Woojin Kim, Seungone |
| contents | Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks remain scarce. We introduce K-BrowseComp, a web-browsing agent benchmark grounded in Korean contexts, consisting of 400 problems. The 300-problem K-BrowseComp-Verified subset is manually constructed and validated by native Korean speakers. On this subset, frontier LLMs, including GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1, reach only 30.00--45.67\%, a substantial drop from BrowseComp, while Korean LLMs released through Korea's Proprietary AI Foundation Model program obtain only 0.00--10.33\%. We further construct a 100-problem synthetic split using hard few-shot exemplars and failure-mode-targeted generation to exploit the asymmetry between solving and creating web browsing problems. On the adversarially filtered synthetic diagnostic split, the strongest model reaches only 26.00\%, and we report this split separately as a targeted stress test. We publicly release our data and code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_02404 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts Lee, Nahyun Yoon, Dongkeun Son, Guijin Kim, Geewook Ko, Dayoon Park, Jeonghun Yoo, Haneul Cho, Jaewon Park, Junghun Lee, Changyoon Jang, Kyochul Kim, Jaeyeon Kim, Eunsu Cho, Woojin Kim, Seungone Computation and Language Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks remain scarce. We introduce K-BrowseComp, a web-browsing agent benchmark grounded in Korean contexts, consisting of 400 problems. The 300-problem K-BrowseComp-Verified subset is manually constructed and validated by native Korean speakers. On this subset, frontier LLMs, including GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1, reach only 30.00--45.67\%, a substantial drop from BrowseComp, while Korean LLMs released through Korea's Proprietary AI Foundation Model program obtain only 0.00--10.33\%. We further construct a 100-problem synthetic split using hard few-shot exemplars and failure-mode-targeted generation to exploit the asymmetry between solving and creating web browsing problems. On the adversarially filtered synthetic diagnostic split, the strongest model reaches only 26.00\%, and we report this split separately as a targeted stress test. We publicly release our data and code. |
| title | K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2606.02404 |