Buchumschlag

PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Vir, Reya, Shankar, Shreya, Chase, Harrison, Fu-Hinthorn, Will, Parameswaran, Aditya
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Computation and Language
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines
von: Shankar, Shreya, et al.
Veröffentlicht: (2024)

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
von: Singhvi, Arnav, et al.
Veröffentlicht: (2023)

RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines
von: Lauro, Quentin Romero, et al.
Veröffentlicht: (2025)

Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
von: Wilcoxson, Max, et al.
Veröffentlicht: (2024)

A Causal Explainable Guardrails for Large Language Models
von: Chu, Zhixuan, et al.
Veröffentlicht: (2024)

"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
von: Shankar, Shreya, et al.
Veröffentlicht: (2024)

Building Guardrails for Large Language Models
von: Dong, Yi, et al.
Veröffentlicht: (2024)

OpenGuardrails: A Configurable, Unified, and Scalable Guardrails Platform for Large Language Models
von: Wang, Thomas, et al.
Veröffentlicht: (2025)

Customize Multi-modal RAI Guardrails with Precedent-based predictions
von: Yang, Cheng-Fu, et al.
Veröffentlicht: (2025)

How Large Language Models Balance Internal Knowledge with User and Document Assertions
von: Li, Shuowei, et al.
Veröffentlicht: (2026)

Trust-Oriented Adaptive Guardrails for Large Language Models
von: Hu, Jinwei, et al.
Veröffentlicht: (2024)

YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models
von: Lin, Junyu, et al.
Veröffentlicht: (2026)

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models
von: Yu, Jiaqi, et al.
Veröffentlicht: (2026)

Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning
von: Ji, Yuelyu, et al.
Veröffentlicht: (2024)

Subliminal Corruption: Mechanisms, Thresholds, and Interpretability
von: Vir, Reya, et al.
Veröffentlicht: (2025)

A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support
von: Nguyen, Long S. T., et al.
Veröffentlicht: (2025)

AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails
von: Chowdhury, Sankalan Pal, et al.
Veröffentlicht: (2024)

AssertBench: A Benchmark for Evaluating Self-Assertion in Large Language Models
von: Lee, Jaeho, et al.
Veröffentlicht: (2025)

Blu-WERP (Web Extraction and Refinement Pipeline): A Scalable Pipeline for Preprocessing Large Language Model Datasets
von: Gowtham, et al.
Veröffentlicht: (2025)

Assertion Enhanced Few-Shot Learning: Instructive Technique for Large Language Models to Generate Educational Explanations
von: Shahriar, Tasmia, et al.
Veröffentlicht: (2023)

SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models
von: Lee, Seanie, et al.
Veröffentlicht: (2025)

Rethinking Dataset Discovery with DataScout
von: Lin, Rachel, et al.
Veröffentlicht: (2025)

SGuard-v1: Safety Guardrail for Large Language Models
von: Lee, JoonHo, et al.
Veröffentlicht: (2025)

SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models
von: Gu, Yiyang, et al.
Veröffentlicht: (2026)

When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
von: Nama, Vihaan, et al.
Veröffentlicht: (2026)

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models
von: Zhao, Jiaxuan, et al.
Veröffentlicht: (2025)

Explaining the Reasoning of Large Language Models Using Attribution Graphs
von: Walker, Chase, et al.
Veröffentlicht: (2025)

Auto-Tuning Safety Guardrails for Black-Box Large Language Models
von: Abdulkadir, Perry
Veröffentlicht: (2025)

Noise Injection Systemically Degrades Large Language Model Safety Guardrails
von: Shahani, Prithviraj Singh, et al.
Veröffentlicht: (2025)

Large Language Models in the Abuse Detection Pipeline
von: Kath, Suraj, et al.
Veröffentlicht: (2026)

Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
von: Zhuang, Jun, et al.
Veröffentlicht: (2025)

BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate
von: Mazza, Arnon, et al.
Veröffentlicht: (2026)

Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees
von: Zeighami, Sepanta, et al.
Veröffentlicht: (2025)

Featurized-Decomposition Join: Low-Cost Semantic Joins with Guarantees
von: Zeighami, Sepanta, et al.
Veröffentlicht: (2025)

Task Cascades for Efficient Unstructured Data Processing
von: Shankar, Shreya, et al.
Veröffentlicht: (2026)

Efficiently Identifying Low-Quality Language Subsets in Multilingual Datasets: A Case Study on a Large-Scale Multilingual Audio Dataset
von: Samir, Farhan, et al.
Veröffentlicht: (2024)

Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets
von: Barbu, Eduard, et al.
Veröffentlicht: (2025)

ML-Bench&Guard: Policy-Grounded Multilingual Safety Benchmark and Guardrail for Large Language Models
von: Zhao, Yunhan, et al.
Veröffentlicht: (2026)

Balanced Multi-Task Attention for Satellite Image Classification: A Systematic Approach to Achieving 97.23% Accuracy on EuroSAT Without Pre-Training
von: Vir, Aditya
Veröffentlicht: (2025)

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters
von: Jin, Haibo, et al.
Veröffentlicht: (2024)