Image de couverture de livre

Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Fang, Yuan, Luo, Yiming, Zhou, Aimin, Tan, Fei
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesis
par: Yang, Qing, et autres
Publié: (2025)

C3AI: Crafting and Evaluating Constitutions for Constitutional AI
par: Kyrychenko, Yara, et autres
Publié: (2025)

Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF
par: Hengle, Amey, et autres
Publié: (2024)

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
par: Lee, Harrison, et autres
Publié: (2023)

Language Models can Evaluate Themselves via Probability Discrepancy
par: Xia, Tingyu, et autres
Publié: (2024)

Inverse Constitutional AI: Compressing Preferences into Principles
par: Findeis, Arduin, et autres
Publié: (2024)

Evaluating the role of `Constitutions' for learning from AI feedback
par: Redgate, Saskia, et autres
Publié: (2024)

SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models
par: Nghiem, Huy, et autres
Publié: (2025)

Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric
par: Koh, Hyukhun, et autres
Publié: (2024)

GloSS over Toxicity: Understanding and Mitigating Toxicity in LLMs via Global Toxic Subspace
par: Duan, Zenghao, et autres
Publié: (2025)

Cause-Aware Empathetic Response Generation via Chain-of-Thought Fine-Tuning
par: Chen, Xinhao, et autres
Publié: (2024)

ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models
par: Wang, Anbang, et autres
Publié: (2024)

Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation
par: Berlin, Konstantin, et autres
Publié: (2026)

ToxiGAN: Toxic Data Augmentation via LLM-Guided Directional Adversarial Generation
par: Li, Peiran, et autres
Publié: (2026)

ConstitutionalExperts: Training a Mixture of Principle-based Prompts
par: Petridis, Savvas, et autres
Publié: (2024)

ToxiLab: How Well Do Open-Source LLMs Generate Synthetic Toxicity Data?
par: Hui, Zheng, et autres
Publié: (2024)

Characterising Toxicity in Generative Large Language Models
par: Zhang, Zhiyao, et autres
Publié: (2026)

Synthesis by Design: Controlled Data Generation via Structural Guidance
par: Xu, Lei, et autres
Publié: (2025)

Kill two birds with one stone: generalized and robust AI-generated text detection via dynamic perturbations
par: Zhou, Yinghan, et autres
Publié: (2025)

Reverse-Engineered Reasoning for Open-Ended Generation
par: Wang, Haozhe, et autres
Publié: (2025)

CIE: Controlling Language Model Text Generations Using Continuous Signals
par: Samuel, Vinay, et autres
Publié: (2025)

Towards Anthropomorphic Conversational AI Part I: A Practical Framework
par: Wei, Fei, et autres
Publié: (2025)

Benchmarking Bengali Dialectal Bias: A Multi-Stage Framework Integrating RAG-Based Translation and Human-Augmented RLAIF
par: Sami, K. M. Jubair, et autres
Publié: (2026)

Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning
par: Zhou, Zhi, et autres
Publié: (2025)

EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content
par: Bi, Shuzhen, et autres
Publié: (2026)

MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation
par: Yang, Zhou, et autres
Publié: (2025)

From Feedback to Checklists: Grounded Evaluation of AI-Generated Clinical Notes
par: Zhou, Karen, et autres
Publié: (2025)

Alignment-Enhanced Decoding:Defending via Token-Level Adaptive Refining of Probability Distributions
par: Liu, Quan, et autres
Publié: (2024)

Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
par: Ning, Yansong, et autres
Publié: (2025)

Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI
par: Maiya, Sharan, et autres
Publié: (2025)

From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGEN
par: Yuan, Peiwen, et autres
Publié: (2025)

FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Models
par: Liu, Shu, et autres
Publié: (2024)

CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models
par: Wagner, Eitan, et autres
Publié: (2024)

MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
par: Fang, Meng, et autres
Publié: (2024)

Literature Meets Data: A Synergistic Approach to Hypothesis Generation
par: Liu, Haokun, et autres
Publié: (2024)

Beyond Toxic: Toxicity Detection Datasets are Not Enough for Brand Safety
par: Korotkova, Elizaveta, et autres
Publié: (2023)

Toxicity Begets Toxicity: Unraveling Conversational Chains in Political Podcasts
par: Rizwan, Naquee, et autres
Publié: (2025)

Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
par: Yuan, Jiahao, et autres
Publié: (2024)

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction
par: Han, Kaiqiao, et autres
Publié: (2024)

COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation
par: Zhou, Tianyi, et autres
Publié: (2026)