Cover Image

AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Bingxiang, Zhang, Wenbin, Song, Jiaxi, Qian, Cheng, Fu, Zixuan, Sun, Bowen, Ding, Ning, Hong, Haiwen, Huang, Longtao, Xue, Hui, Cui, Ganqu, Che, Wanxiang, Liu, Zhiyuan, Sun, Maosong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
by: He, Bingxiang, et al.
Published: (2024)

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
by: Guo, Yiju, et al.
Published: (2024)

Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models
by: Ding, Ning, et al.
Published: (2024)

UltraFeedback: Boosting Language Models with Scaled AI Feedback
by: Cui, Ganqu, et al.
Published: (2023)

Advancing LLM Reasoning Generalists with Preference Trees
by: Yuan, Lifan, et al.
Published: (2024)

CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning
by: Li, Ran, et al.
Published: (2026)

From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones
by: Yuan, Lifan, et al.
Published: (2025)

Empowering Private Tutoring by Chaining Large Language Models
by: Chen, Yulin, et al.
Published: (2023)

JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
by: He, Bingxiang, et al.
Published: (2025)

UltraIF: Advancing Instruction Following from the Wild
by: An, Kaikai, et al.
Published: (2025)

AIR: Improving Agent Safety through Incident Response
by: Xiao, Zibo, et al.
Published: (2026)

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs
by: Gao, Cheng, et al.
Published: (2024)

Representation Learning for Natural Language Processing
by: Liu, Zhiyuan, et al.
Published: (2020)

Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning
by: Yang, Sen, et al.
Published: (2024)

How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
by: Xu, Ziwen, et al.
Published: (2026)

Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets
by: Feng, Duanyu, et al.
Published: (2024)

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
by: Lv, Xingtai, et al.
Published: (2024)

Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models
by: Chen, Hao, et al.
Published: (2026)

How Far Can Unsupervised RLVR Scale LLM Training?
by: He, Bingxiang, et al.
Published: (2026)

RLPR: Extrapolating RLVR to General Domains without Verifiers
by: Yu, Tianyu, et al.
Published: (2025)

Beyond Pairs: Your Language Model is Secretly Optimizing a Preference Graph
by: Liu, Ning, et al.
Published: (2026)

Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
by: Wang, Xing, et al.
Published: (2025)

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
by: Qian, Cheng, et al.
Published: (2024)

AnnoDPO: Protein Functional Annotation Learning with Direct Preference Optimization
by: Jiang, Zixuan, et al.
Published: (2025)

ESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling
by: Teng, Dechuan, et al.
Published: (2026)

AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMs
by: Li, Shangzhan, et al.
Published: (2025)

Free Process Rewards without Process Labels
by: Yuan, Lifan, et al.
Published: (2024)

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing
by: Zhang, Kai, et al.
Published: (2023)

States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly
by: Chen, Junhao, et al.
Published: (2024)

INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair
by: Wang, Hanbin, et al.
Published: (2023)

M3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Following
by: Gao, Ruirui, et al.
Published: (2025)

Exploring Format Consistency for Instruction Tuning
by: Liang, Shihao, et al.
Published: (2023)

RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
by: Yu, Tianyu, et al.
Published: (2023)

ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation
by: Chen, Hao, et al.
Published: (2025)

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
by: Chen, Weize, et al.
Published: (2024)

Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotations
by: Cui, Yibo, et al.
Published: (2025)

Exploring Multi-Grained Concept Annotations for Multimodal Large Language Models
by: Xu, Xiao, et al.
Published: (2024)

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
by: Wang, Hanqing, et al.
Published: (2024)

TC-Padé: Trajectory-Consistent Padé Approximation for Diffusion Acceleration
by: Cui, Benlei, et al.
Published: (2026)

AFD-INSTRUCTION: A Comprehensive Antibody Instruction Dataset with Functional Annotations for LLM-Based Understanding and Design
by: Luo, Ling, et al.
Published: (2026)