Cover Image

Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Burnat, Florian A. D., Davidson, Brittany I.
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandates
by: Burnat, Florian A. D., et al.
Published: (2025)

Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
by: Ruan, Kai, et al.
Published: (2024)

Bring Your Own Prompts: Use-Case-Specific Bias and Fairness Evaluation for LLMs
by: Bouchard, Dylan
Published: (2024)

Regulatory gray areas of LLM Terms
by: Davidson, Brittany I., et al.
Published: (2026)

Narrative Context Protocol: An Open-Source Storytelling Framework for Generative AI
by: Gerba, Hank
Published: (2025)

Context Misleads LLMs: The Role of Context Filtering in Maintaining Safe Alignment of LLMs
by: Kim, Jinhwa, et al.
Published: (2025)

Qworld: Question-Specific Evaluation Criteria for LLMs
by: Gao, Shanghua, et al.
Published: (2026)

Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
by: Trott, Sean, et al.
Published: (2026)

Evaluating the Sensitivity of LLMs to Prior Context
by: Hankache, Robert, et al.
Published: (2025)

Tamper-Resistant Safeguards for Open-Weight LLMs
by: Tamirisa, Rishub, et al.
Published: (2024)

DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models
by: Kim, Olivia
Published: (2025)

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
by: Xu, Zhangchen, et al.
Published: (2024)

Humans and LLMs Diverge on Probabilistic Inferences
by: Kamath, Gaurav, et al.
Published: (2026)

Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Semantic Alignment
by: Sun, Yanru, et al.
Published: (2025)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
by: Thakur, Aman Singh, et al.
Published: (2024)

Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in
by: Agarwal, Utkarsh, et al.
Published: (2024)

Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions
by: Murugadoss, Bhuvanashree, et al.
Published: (2024)

SDA: Steering-Driven Distribution Alignment for Open LLMs without Fine-Tuning
by: Xia, Wei, et al.
Published: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
by: Liu, Yixin, et al.
Published: (2025)

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs
by: Guttmann, Kamil, et al.
Published: (2026)

Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs
by: Tan, Wenhui, et al.
Published: (2026)

Systematic Evaluation of Long-Context LLMs on Financial Concepts
by: Gupta, Lavanya, et al.
Published: (2024)

ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation
by: Zheng, Jingnan, et al.
Published: (2024)

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment
by: Huang, Heyan, et al.
Published: (2024)

TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes
by: Shah, Raj Sanjay, et al.
Published: (2025)

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation
by: Li, Mufei, et al.
Published: (2025)

Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs
by: Zhang, Yifang, et al.
Published: (2025)

Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
by: Bazarova, Alexandra, et al.
Published: (2025)

Unveiling Divergent Inductive Biases of LLMs on Temporal Data
by: Kishore, Sindhu, et al.
Published: (2024)

Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines
by: Song, Hwanjun
Published: (2026)

LLM-Human Pipeline for Cultural Context Grounding of Conversations
by: Pujari, Rajkumar, et al.
Published: (2024)

Pipelined Decoder for Efficient Context-Aware Text Generation
by: Huang, Zixian, et al.
Published: (2025)

Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation
by: Zhang, Xiaoying, et al.
Published: (2024)

From Guidelines to Guarantees: A Graph-Based Evaluation Harness for Domain-Specific Evaluation of LLMs
by: Lundin, Jessica M., et al.
Published: (2025)

Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning
by: Chen, Po-Chun, et al.
Published: (2026)

OpenSanctions Pairs: Large-Scale Entity Matching with LLMs
by: Smith, Chandler, et al.
Published: (2026)

StealthGraph: Exposing Domain-Specific Risks in LLMs through Knowledge-Graph-Guided Harmful Prompt Generation
by: Zheng, Huawei, et al.
Published: (2026)

Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs
by: Singhal, Ronit, et al.
Published: (2024)

Localizing AI: Evaluating Open-Weight Language Models for Languages of Baltic States
by: Kapočiūtė-Dzikienė, Jurgita, et al.
Published: (2025)

Quotient Semivalues for False-Name-Resistant Data Attribution
by: Burnat, Florian A. D., et al.
Published: (2026)