Cover Image

Eye of Judgement: Dissecting the Evaluation of Russian-speaking LLMs with POLLUX

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Martynov, Nikita, Mordasheva, Anastasia, Gorbetskiy, Dmitriy, Astafurov, Danil, Isaeva, Ulyana, Basyrova, Elina, Skachkov, Sergey, Berestova, Victoria, Ivanov, Nikolay, Zanina, Valeriia, Fenogenova, Alena
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MERA: A Comprehensive LLM Evaluation in Russian
by: Fenogenova, Alena, et al.
Published: (2024)

A Family of Pretrained Transformer Language Models for Russian
by: Zmitrovich, Dmitry, et al.
Published: (2023)

The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design
by: Snegirev, Artem, et al.
Published: (2024)

REPA: Russian Error Types Annotation for Evaluating Text Generation and Judgment Capabilities
by: Pugachev, Alexander, et al.
Published: (2025)

Multimodal Evaluation of Russian-language Architectures
by: Chervyakov, Artem, et al.
Published: (2025)

RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs
by: Taktasheva, Ekaterina, et al.
Published: (2024)

FiMMIA: scaling semantic perturbation-based membership inference across modalities
by: Emelyanov, Anton, et al.
Published: (2025)

RusLICA: A Russian-Language Platform for Automated Linguistic Inquiry and Category Analysis
by: Sigdel, Elina, et al.
Published: (2026)

Long Input Benchmark for Russian Analysis
by: Churin, Igor, et al.
Published: (2024)

Russian Pendulum
by: Ledeneva, Alena
Published: (2026)

Contemporary Russian Literature in Latvia: Children’s Literature
by: Elina Vasiljeva
Published: (2022)

GigaEmbeddings: Efficient Russian Language Embedding Model
by: Kolodin, Egor, et al.
Published: (2025)

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
by: Adamenko, Pavel, et al.
Published: (2025)

Russian Wheat Export Market Dataset: FOB Prices, CBOT Futures, Exchange Rates, and Grain Damper Export Duties (2018–2024)
by: Lakerbai, Rebekka-Valeriia, et al.
Published: (2026)

Russian-Language Multimodal Dataset for Automatic Summarization of Scientific Papers
by: Tsanda, Alena, et al.
Published: (2024)

CoRoVA: Compressed Representations for Vector-Augmented Code Completion
by: Cherniuk, Daria, et al.
Published: (2025)

Maximum Impact with Fewer Features: Efficient Feature Selection for Cold-Start Recommenders through Collaborative Importance Weighting
by: Sukhorukov, Nikita, et al.
Published: (2025)

Recommendation Is a Dish Better Served Warm
by: Gusak, Danil, et al.
Published: (2025)

Polymorphism of Avr Genes in Russian Populations of Phytophthora infestans
by: Vera K. Chizhik, et al.
Published: (2024)

Video in Russian - The Phenomenon of Collective Memory in the Semantic Contexts of Post-Modernity
by: Anastasia Nenița
Published: (2025)

DRAGOn: Designing RAG On Periodically Updated Corpus
by: Chernogorskii, Fedor, et al.
Published: (2025)

Notes on Images and Communication
by: Martynov, Denis
Published: (2026)

VoXtream2: Full-stream TTS with dynamic speaking rate control
by: Torgashov, Nikita, et al.
Published: (2026)

Qwerty AI: Explainable Automated Age Rating and Content Safety Assessment for Russian-Language Screenplays
by: Zmanovskii, Nikita
Published: (2025)

Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies
by: Matkin, Nikita, et al.
Published: (2024)

Russian Financial Statements Database: A firm-level collection of the universe of financial statements
by: Bondarkov, Sergey, et al.
Published: (2025)

On-chip high-order parametric downconversion in the excitonic Mott insulator Nb$_3$Cl$_8$ for programmable multiphoton entangled states
by: Skachkov, Dmitry, et al.
Published: (2025)

Linear and nonlinear optical response based on many-body GW-Bethe-Salpeter and Kadanoff-Baym approaches for two-dimensional layered semiconductors
by: Skachkov, Dmitry, et al.
Published: (2024)

Learning-Augmented Online Caching: New Upper Bounds
by: Skachkov, Daniel, et al.
Published: (2024)

On finding formal power-logarithmic expansions of solutions to $q$-difference equations
by: Gaianov, Nikita, et al.
Published: (2025)

Linearized $\mathcal{N}=2$ conformal supergravity in the harmonic approach
by: Ivanov, Evgeny, et al.
Published: (2025)

$\mathcal{N}=2$ superconformal gravitino in harmonic superspace
by: Ivanov, Evgeny, et al.
Published: (2024)

Off-shell invariants of linearized $4D, \mathcal{N}=2$ supergravity in the harmonic approach
by: Ivanov, Evgeny, et al.
Published: (2024)

$\mathcal{N}=2$ AdS hypermultiplets in harmonic superspace
by: Ivanov, Evgeny, et al.
Published: (2025)

BiRQA: Bidirectional Robust Quality Assessment for Images
by: Gushchin, Aleksandr, et al.
Published: (2026)

IOI: Invisible One-Iteration Adversarial Attack on No-Reference Image- and Video-Quality Metrics
by: Shumitskaya, Ekaterina, et al.
Published: (2024)

FS-IQA: Certified Feature Smoothing for Robust Image Quality Assessment
by: Shumitskaya, Ekaterina, et al.
Published: (2025)

Computational lower bounds for multi-frequency group synchronization
by: Kireeva, Anastasia, et al.
Published: (2024)

Semantic Shifts of Psychological Concepts in Scientific and Popular Media Discourse: A Distributional Semantics Analysis of Russian-Language Corpora
by: Anastasia, Orlova
Published: (2026)

In the Blink of an Eye: Instant Game Map Editing using a Generative-AI Smart Brush
by: Gnatyuk, Vitaly, et al.
Published: (2025)