Cover Image

PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vir, Reya, Shankar, Shreya, Chase, Harrison, Fu-Hinthorn, Will, Parameswaran, Aditya
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines
by: Shankar, Shreya, et al.
Published: (2024)

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
by: Singhvi, Arnav, et al.
Published: (2023)

RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines
by: Lauro, Quentin Romero, et al.
Published: (2025)

Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
by: Wilcoxson, Max, et al.
Published: (2024)

A Causal Explainable Guardrails for Large Language Models
by: Chu, Zhixuan, et al.
Published: (2024)

"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
by: Shankar, Shreya, et al.
Published: (2024)

Building Guardrails for Large Language Models
by: Dong, Yi, et al.
Published: (2024)

OpenGuardrails: A Configurable, Unified, and Scalable Guardrails Platform for Large Language Models
by: Wang, Thomas, et al.
Published: (2025)

Customize Multi-modal RAI Guardrails with Precedent-based predictions
by: Yang, Cheng-Fu, et al.
Published: (2025)

How Large Language Models Balance Internal Knowledge with User and Document Assertions
by: Li, Shuowei, et al.
Published: (2026)

Trust-Oriented Adaptive Guardrails for Large Language Models
by: Hu, Jinwei, et al.
Published: (2024)

YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models
by: Lin, Junyu, et al.
Published: (2026)

SentGuard: Sentence-Level Streaming Guardrails for Large Language Models
by: Yu, Jiaqi, et al.
Published: (2026)

Assertion Detection Large Language Model In-context Learning LoRA Fine-tuning
by: Ji, Yuelyu, et al.
Published: (2024)

Subliminal Corruption: Mechanisms, Thresholds, and Interpretability
by: Vir, Reya, et al.
Published: (2025)

A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support
by: Nguyen, Long S. T., et al.
Published: (2025)

AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails
by: Chowdhury, Sankalan Pal, et al.
Published: (2024)

AssertBench: A Benchmark for Evaluating Self-Assertion in Large Language Models
by: Lee, Jaeho, et al.
Published: (2025)

Blu-WERP (Web Extraction and Refinement Pipeline): A Scalable Pipeline for Preprocessing Large Language Model Datasets
by: Gowtham, et al.
Published: (2025)

Assertion Enhanced Few-Shot Learning: Instructive Technique for Large Language Models to Generate Educational Explanations
by: Shahriar, Tasmia, et al.
Published: (2023)

SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models
by: Lee, Seanie, et al.
Published: (2025)

Rethinking Dataset Discovery with DataScout
by: Lin, Rachel, et al.
Published: (2025)

SGuard-v1: Safety Guardrail for Large Language Models
by: Lee, JoonHo, et al.
Published: (2025)

SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models
by: Gu, Yiyang, et al.
Published: (2026)

When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
by: Nama, Vihaan, et al.
Published: (2026)

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models
by: Zhao, Jiaxuan, et al.
Published: (2025)

Explaining the Reasoning of Large Language Models Using Attribution Graphs
by: Walker, Chase, et al.
Published: (2025)

Auto-Tuning Safety Guardrails for Black-Box Large Language Models
by: Abdulkadir, Perry
Published: (2025)

Noise Injection Systemically Degrades Large Language Model Safety Guardrails
by: Shahani, Prithviraj Singh, et al.
Published: (2025)

Large Language Models in the Abuse Detection Pipeline
by: Kath, Suraj, et al.
Published: (2026)

Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
by: Zhuang, Jun, et al.
Published: (2025)

BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate
by: Mazza, Arnon, et al.
Published: (2026)

Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees
by: Zeighami, Sepanta, et al.
Published: (2025)

Featurized-Decomposition Join: Low-Cost Semantic Joins with Guarantees
by: Zeighami, Sepanta, et al.
Published: (2025)

Task Cascades for Efficient Unstructured Data Processing
by: Shankar, Shreya, et al.
Published: (2026)

Efficiently Identifying Low-Quality Language Subsets in Multilingual Datasets: A Case Study on a Large-Scale Multilingual Audio Dataset
by: Samir, Farhan, et al.
Published: (2024)

Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets
by: Barbu, Eduard, et al.
Published: (2025)

ML-Bench&Guard: Policy-Grounded Multilingual Safety Benchmark and Guardrail for Large Language Models
by: Zhao, Yunhan, et al.
Published: (2026)

Balanced Multi-Task Attention for Satellite Image Classification: A Systematic Approach to Achieving 97.23% Accuracy on EuroSAT Without Pre-Training
by: Vir, Aditya
Published: (2025)

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters
by: Jin, Haibo, et al.
Published: (2024)