Copertina

Counting-Stars: A Multi-evidence, Position-aware, and Scalable Benchmark for Evaluating Long-Context Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Song, Mingyang, Zheng, Mao, Luo, Xuan
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Study
di: Song, Mingyang, et al.
Pubblicazione: (2024)

A Survey of Query Optimization in Large Language Models
di: Song, Mingyang, et al.
Pubblicazione: (2024)

GRP: Goal-Reversed Prompting for Zero-Shot Evaluation with LLMs
di: Song, Mingyang, et al.
Pubblicazione: (2025)

Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions
di: Song, Mingyang, et al.
Pubblicazione: (2026)

A Survey of On-Policy Distillation for Large Language Models
di: Song, Mingyang, et al.
Pubblicazione: (2026)

Mil-SCORE: Benchmarking Long-Context Geospatial Reasoning and Planning in Large Language Models
di: Palnitkar, Aadi, et al.
Pubblicazione: (2026)

MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoning
di: Li, Zheng, et al.
Pubblicazione: (2024)

FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models
di: Song, Mingyang, et al.
Pubblicazione: (2025)

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
di: Dai, Chang, et al.
Pubblicazione: (2025)

CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models
di: Qiu, Zexuan, et al.
Pubblicazione: (2024)

PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generation
di: Xu, Chenning, et al.
Pubblicazione: (2026)

M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
di: Kwan, Wai-Chung, et al.
Pubblicazione: (2023)

Beyond the Illusion of Consensus: From Surface Heuristics to Knowledge-Grounded Evaluation in LLM-as-a-Judge
di: Song, Mingyang, et al.
Pubblicazione: (2026)

Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning
di: Song, Mingyang, et al.
Pubblicazione: (2025)

ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage
di: Lee, Taewhoo, et al.
Pubblicazione: (2024)

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
di: Chen, Longze, et al.
Pubblicazione: (2024)

MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models
di: Huang, Zhongzhan, et al.
Pubblicazione: (2025)

ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models
di: Chen, Haibin, et al.
Pubblicazione: (2025)

LongSafety: Evaluating Long-Context Safety of Large Language Models
di: Lu, Yida, et al.
Pubblicazione: (2025)

BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
di: Luo, Kun, et al.
Pubblicazione: (2024)

CodeDelegator: Mitigating Context Pollution via Role Separation in Code-as-Action Agents
di: Fei, Tianxiang, et al.
Pubblicazione: (2026)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
di: Hu, Xiang, et al.
Pubblicazione: (2025)

Thus Spake Long-Context Large Language Model
di: Liu, Xiaoran, et al.
Pubblicazione: (2025)

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
di: Ye, Xi, et al.
Pubblicazione: (2025)

Evaluating Large Language Models on Spatial Tasks: A Multi-Task Benchmarking Study
di: Xu, Liuchang, et al.
Pubblicazione: (2024)

LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts
di: Chen, Junhao, et al.
Pubblicazione: (2025)

MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
di: Zhou, Keyan, et al.
Pubblicazione: (2025)

IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
di: Sun, Mingrui, et al.
Pubblicazione: (2026)

LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction
di: Liu, Weichu, et al.
Pubblicazione: (2025)

SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
di: Puvvada, Krishna C., et al.
Pubblicazione: (2025)

PRISM: Probability Reallocation with In-Span Masking for Knowledge-Sensitive Alignment
di: Xu, Chenning, et al.
Pubblicazione: (2026)

LongReasonArena: A Long Reasoning Benchmark for Large Language Models
di: Ding, Jiayu, et al.
Pubblicazione: (2025)

DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models
di: Tiwari, Utkarsh, et al.
Pubblicazione: (2025)

Marathon: A Race Through the Realm of Long Context with Large Language Models
di: Zhang, Lei, et al.
Pubblicazione: (2023)

Context-aware Rotary Position Embedding
di: Veisi, Ali, et al.
Pubblicazione: (2025)

SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
di: Yu, Jing, et al.
Pubblicazione: (2025)

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models
di: Dong, Zican, et al.
Pubblicazione: (2023)

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
di: Wang, Lanrui, et al.
Pubblicazione: (2025)

Ebisu: Benchmarking Large Language Models in Japanese Finance
di: Peng, Xueqing, et al.
Pubblicazione: (2026)

Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding
di: Zhang, Zhihan, et al.
Pubblicazione: (2024)