Copertina

A Single Character can Make or Break Your LLM Evals

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Su, Jingtong, Zhang, Jianyu, Ullrich, Karen, Bottou, Léon, Ibrahim, Mark
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
di: Su, Jingtong, et al.
Pubblicazione: (2024)

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers
di: Su, Jingtong, et al.
Pubblicazione: (2025)

Memory Mosaics at scale
di: Zhang, Jianyu, et al.
Pubblicazione: (2025)

Make Your LLM Fully Utilize the Context
di: An, Shengnan, et al.
Pubblicazione: (2024)

Dual Optimal: Make Your LLM Peer-like with Dignity
di: Wang, Xiangqi, et al.
Pubblicazione: (2026)

These Are Not All the Features You Are Looking For: A Fundamental Bottleneck in Supervised Pretraining
di: Yang, Xingyu Alice, et al.
Pubblicazione: (2025)

CHILL at SemEval-2025 Task 2: You Can't Just Throw Entities and Hope -- Make Your LLM to Get Them Right
di: Lee, Jaebok, et al.
Pubblicazione: (2025)

Memory Mosaics
di: Zhang, Jianyu, et al.
Pubblicazione: (2024)

EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agents
di: Fish, Sara, et al.
Pubblicazione: (2025)

TruthEval: A Dataset to Evaluate LLM Truthfulness and Reliability
di: Khatun, Aisha, et al.
Pubblicazione: (2024)

SurveyEval: Towards Comprehensive Evaluation of LLM-Generated Academic Surveys
di: Zhao, Jiahao, et al.
Pubblicazione: (2025)

TimeStampEval: A Simple LLM Eval and a Little Fuzzy Matching Trick to Improve Search Accuracy
di: McCammon, James
Pubblicazione: (2025)

Is Your LLM Outdated? A Deep Look at Temporal Generalization
di: Zhu, Chenghao, et al.
Pubblicazione: (2024)

Single Character Perturbations Break LLM Alignment
di: Lin, Leon, et al.
Pubblicazione: (2024)

Enhancing LLM Character-Level Manipulation via Divide and Conquer
di: Xiong, Zhen, et al.
Pubblicazione: (2025)

CREFT: Sequential Multi-Agent LLM for Character Relation Extraction
di: Chun, Ye Eun, et al.
Pubblicazione: (2025)

AcademicEval: Live Long-Context LLM Benchmark
di: Zhang, Haozhen, et al.
Pubblicazione: (2025)

Measuring all the noises of LLM Evals
di: Wang, Sida
Pubblicazione: (2025)

YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
di: D'Souza, Jennifer, et al.
Pubblicazione: (2025)

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
di: Yang, Langqi, et al.
Pubblicazione: (2025)

ZeroSumEval: Scaling LLM Evaluation with Inter-Model Competition
di: Khan, Haidar, et al.
Pubblicazione: (2025)

ViLLM-Eval: A Comprehensive Evaluation Suite for Vietnamese Large Language Models
di: Nguyen, Trong-Hieu, et al.
Pubblicazione: (2024)

Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
di: Zhou, Cai, et al.
Pubblicazione: (2025)

DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer
di: Hong, Junyuan, et al.
Pubblicazione: (2023)

Collaborative Quest Completion with LLM-driven Non-Player Characters in Minecraft
di: Rao, Sudha, et al.
Pubblicazione: (2024)

Experiences Build Characters: The Linguistic Origins and Functional Impact of LLM Personality
di: Wang, Xi, et al.
Pubblicazione: (2026)

Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
di: Liu, Ryan, et al.
Pubblicazione: (2024)

LexInstructEval: Lexical Instruction Following Evaluation for Large Language Models
di: Ren, Huimin, et al.
Pubblicazione: (2025)

AIC CTU@FEVER 8: On-premise fact checking through long context RAG
di: Ullrich, Herbert, et al.
Pubblicazione: (2025)

Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning
di: Silver, Sam, et al.
Pubblicazione: (2025)

ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition
di: Alyahya, Hisham A., et al.
Pubblicazione: (2025)

AutoEvoEval: An Automated Framework for Evolving Close-Ended LLM Evaluation Data
di: Wu, JiaRu, et al.
Pubblicazione: (2025)

ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
di: Wu, Kevin, et al.
Pubblicazione: (2024)

CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation
di: Zhao, Jingqian, et al.
Pubblicazione: (2025)

Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmark
di: Xu, Shuhang, et al.
Pubblicazione: (2025)

Deflanderization for Game Dialogue: Balancing Character Authenticity with Task Execution in LLM-based NPCs
di: Buakhaw, Pasin, et al.
Pubblicazione: (2025)

Understanding and Mitigating Tokenization Bias in Language Models
di: Phan, Buu, et al.
Pubblicazione: (2024)

Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction
di: Chen, Xingwu, et al.
Pubblicazione: (2026)

Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts
di: Luo, Sijia, et al.
Pubblicazione: (2026)

Who can we trust? LLM-as-a-jury for Comparative Assessment
di: Qian, Mengjie, et al.
Pubblicazione: (2026)