Imagen de Portada

ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhuo, Jingming, Zhang, Songyang, Fang, Xinyu, Duan, Haodong, Lin, Dahua, Chen, Kai
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
por: Wang, Chonghua, et al.
Publicado: (2024)

Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs
por: Qiao, Yuxuan, et al.
Publicado: (2024)

MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark
por: Liu, Hongwei, et al.
Publicado: (2024)

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step
por: Chen, Zehui, et al.
Publicado: (2023)

NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities
por: Li, Mo, et al.
Publicado: (2024)

CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolution
por: Cao, Maosong, et al.
Publicado: (2024)

CIBench: Evaluating Your LLMs with a Code Interpreter Plugin
por: Zhang, Chuyu, et al.
Publicado: (2024)

MMBench: Is Your Multi-modal Model an All-around Player?
por: Liu, Yuan, et al.
Publicado: (2023)

Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs
por: Gu, Yuzhe, et al.
Publicado: (2025)

MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
por: Fang, Xinyu, et al.
Publicado: (2024)

Understanding the Prompt Sensitivity
por: Liu, Yang, et al.
Publicado: (2026)

The Imitation Game: Turing Machine Imitator is Length Generalizable Reasoner
por: Hua, Zhouqi, et al.
Publicado: (2025)

Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement
por: Cao, Maosong, et al.
Publicado: (2025)

Are Your LLMs Capable of Stable Reasoning?
por: Liu, Junnan, et al.
Publicado: (2024)

Coding Triangle: How Does Large Language Model Understand Code?
por: Zhang, Taolin, et al.
Publicado: (2025)

Think Visually, Reason Textually: Vision-Language Synergy in ARC
por: Zhang, Beichen, et al.
Publicado: (2025)

Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia
por: Zhou, Zhejian, et al.
Publicado: (2024)

Redundancy Principles for MLLMs Benchmarks
por: Zhang, Zicheng, et al.
Publicado: (2025)

CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
por: Liu, Shudong, et al.
Publicado: (2025)

The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs
por: Guan, Bryan, et al.
Publicado: (2025)

WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
por: Ding, Shuangrui, et al.
Publicado: (2026)

InternLM-Law: An Open Source Chinese Legal Large Language Model
por: Fei, Zhiwei, et al.
Publicado: (2024)

LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models
por: Chen, Xi, et al.
Publicado: (2024)

LEAN-GitHub: Compiling GitHub LEAN repositories for a versatile LEAN prover
por: Wu, Zijian, et al.
Publicado: (2024)

ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs
por: Thomas, Rohan Subramanian, et al.
Publicado: (2026)

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM
por: Fang, Xinyu, et al.
Publicado: (2025)

Rectifying LLM Thought from Lens of Optimization
por: Liu, Junnan, et al.
Publicado: (2025)

The Biased Oracle: Assessing LLMs' Understandability and Empathy in Medical Diagnoses
por: Yao, Jianzhou, et al.
Publicado: (2025)

InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model
por: Zang, Yuhang, et al.
Publicado: (2025)

Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement
por: Zhan, Pengwei, et al.
Publicado: (2024)

Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains
por: Yuan, Zhonghang, et al.
Publicado: (2026)

BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning
por: Zhang, Beichen, et al.
Publicado: (2025)

Echotune: A Modular Extractor Leveraging the Variable-Length Nature of Speech in ASR Tasks
por: Chen, Sizhou, et al.
Publicado: (2023)

P3: Prompts Promote Prompting
por: Zhang, Xinyu, et al.
Publicado: (2025)

PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities
por: Sravanthi, Settaluri Lakshmi, et al.
Publicado: (2024)

F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods
por: Sun, Yu, et al.
Publicado: (2024)

Exploring the Sensitivity of LLMs' Decision-Making Capabilities: Insights from Prompt Variation and Hyperparameters
por: Loya, Manikanta, et al.
Publicado: (2023)

Understanding the Ability of LLMs to Handle Character-Level Perturbation
por: Zhuo, Anyuan, et al.
Publicado: (2025)

Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
por: Hua, Andong, et al.
Publicado: (2025)

MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
por: Fu, Chaoyou, et al.
Publicado: (2024)