Cover Image

What's In My Human Feedback? Learning Interpretable Descriptions of Preference Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Movva, Rajiv, Milli, Smitha, Min, Sewon, Pierson, Emma
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse Autoencoders for Hypothesis Generation
by: Movva, Rajiv, et al.
Published: (2025)

Use Sparse Autoencoders to Discover Unknown Concepts, Not to Act on Known Concepts
by: Peng, Kenny, et al.
Published: (2025)

Annotation alignment: Comparing LLM and human annotations of conversational safety
by: Movva, Rajiv, et al.
Published: (2024)

Model-based Preference Optimization in Abstractive Summarization without Human Feedback
by: Choi, Jaepill, et al.
Published: (2024)

Learning to Detect Language Model Training Data via Active Reconstruction
by: Yin, Junjie Oscar, et al.
Published: (2026)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
by: Cao, Qingqing, et al.
Published: (2023)

Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning
by: Poddar, Sriyash, et al.
Published: (2024)

Generative AI in Medicine
by: Shanmugam, Divya, et al.
Published: (2024)

Topics, Authors, and Institutions in Large Language Model Research: Trends from 17K arXiv Papers
by: Movva, Rajiv, et al.
Published: (2023)

LRHP: Learning Representations for Human Preferences via Preference Pairs
by: Wang, Chenglong, et al.
Published: (2024)

ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
by: Melikidze, Davit, et al.
Published: (2026)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

What's New in My Data? Novelty Exploration via Contrastive Generation
by: Isonuma, Masaru, et al.
Published: (2024)

Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback
by: Lerner, Emilia Agis, et al.
Published: (2024)

RAG over Thinking Traces Can Improve Reasoning Tasks
by: Arabzadeh, Negar, et al.
Published: (2026)

Reasoning Models Can Be Effective Without Thinking
by: Ma, Wenjie, et al.
Published: (2025)

Aligning LLMs through Multi-perspective User Preference Ranking-based Feedback for Programming Question Answering
by: Yang, Hongyu, et al.
Published: (2024)

Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
by: Lee, Chia-Hsuan, et al.
Published: (2026)

Dissecting Human and LLM Preferences
by: Li, Junlong, et al.
Published: (2024)

Rethinking Diverse Human Preference Learning through Principal Component Analysis
by: Luo, Feng, et al.
Published: (2025)

Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
by: Ma, Xilai, et al.
Published: (2026)

Improving Reinforcement Learning from Human Feedback Using Contrastive Rewards
by: Shen, Wei, et al.
Published: (2024)

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization
by: Dong, Zhijin
Published: (2025)

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
by: Liu, Jiacheng, et al.
Published: (2024)

What's Wrong? Refining Meeting Summaries with LLM Feedback
by: Kirstein, Frederic, et al.
Published: (2024)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

Can LLMs Capture Human Preferences?
by: Goli, Ali, et al.
Published: (2023)

Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization
by: Lu, Junjie, et al.
Published: (2025)

Unsupervised Human Preference Learning
by: Shashidhar, Sumuk, et al.
Published: (2024)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
by: Zhang, Xuemiao, et al.
Published: (2025)

Is On-Policy Data always the Best Choice for Direct Preference Optimization-based LM Alignment?
by: Sun, Zetian, et al.
Published: (2025)

What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
by: Yang, Chenyang, et al.
Published: (2024)

Distributionally Robust Reinforcement Learning with Human Feedback
by: Mandal, Debmalya, et al.
Published: (2025)

Learning Personalized Agents from Human Feedback
by: Liang, Kaiqu, et al.
Published: (2026)

Rethinking Human Preference Evaluation of LLM Rationales
by: Li, Ziang, et al.
Published: (2025)

Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI
by: Wang, Yuxia, et al.
Published: (2025)

MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
by: Li, Shuyue Stella, et al.
Published: (2024)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
by: Xiong, Chen, et al.
Published: (2025)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
by: Wang, Zhilin, et al.
Published: (2025)

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
by: Lee, Harrison, et al.
Published: (2023)