Copertina

Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xu, Guangxuan, Xu, Kai, Sudalairaj, Shivchander, Wang, Hao, Srivastava, Akash
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methods
di: Puri, Isha, et al.
Pubblicazione: (2025)

Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling
di: Giannone, Giorgio, et al.
Pubblicazione: (2025)

LAB: Large-Scale Alignment for ChatBots
di: Sudalairaj, Shivchander, et al.
Pubblicazione: (2024)

Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
di: Pareja, Aldo, et al.
Pubblicazione: (2024)

S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
di: Han, Ligong, et al.
Pubblicazione: (2026)

Instruction-Tuning LLMs for Event Extraction with Annotation Guidelines
di: Srivastava, Saurabh, et al.
Pubblicazione: (2025)

SQuat: Subspace-orthogonal KV Cache Quantization
di: Wang, Hao, et al.
Pubblicazione: (2025)

Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
di: Nayak, Nikhil Shivakumar, et al.
Pubblicazione: (2025)

ReasonScaffold: A Scaffolded Reasoning-based Annotation Protocol for Human-AI Co-Annotation
di: Sudheendra, Smitha Muthya, et al.
Pubblicazione: (2026)

VRM: Teaching Reward Models to Understand Authentic Human Preferences
di: Liu, Biao, et al.
Pubblicazione: (2026)

Tokenization Preference for Human and Machine Learning Model: An Annotation Study
di: Hiraoka, Tatsuya, et al.
Pubblicazione: (2023)

LLMs as Span Annotators: A Comparative Study of LLMs and Humans
di: Kasner, Zdeněk, et al.
Pubblicazione: (2025)

Reducing Hallucinations in LLMs via Factuality-Aware Preference Learning
di: Chaduvula, Sindhuja, et al.
Pubblicazione: (2026)

Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs
di: Liu, Fan, et al.
Pubblicazione: (2024)

BitDelta: Your Fine-Tune May Only Be Worth One Bit
di: Liu, James, et al.
Pubblicazione: (2024)

Evaluating and Aligning CodeLLMs on Human Preference
di: Yang, Jian, et al.
Pubblicazione: (2024)

Can LLMs Capture Human Preferences?
di: Goli, Ali, et al.
Pubblicazione: (2023)

KDCM: Reducing Hallucination in LLMs through Explicit Reasoning Structures
di: Hao, Jinbo, et al.
Pubblicazione: (2026)

The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
di: Verma, Nikhil, et al.
Pubblicazione: (2025)

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization
di: Sun, Huashan, et al.
Pubblicazione: (2025)

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
di: Chiang, Wei-Lin, et al.
Pubblicazione: (2024)

Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization
di: Chen, Beiduo, et al.
Pubblicazione: (2026)

LLMs as Data Annotators: How Close Are We to Human Performance
di: Haq, Muhammad Uzair Ul, et al.
Pubblicazione: (2025)

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
di: Wang, Peiyi, et al.
Pubblicazione: (2023)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

A Survey on Human-Centric LLMs
di: Wang, Jing Yi, et al.
Pubblicazione: (2024)

BATON: Aligning Text-to-Audio Model with Human Preference Feedback
di: Liao, Huan, et al.
Pubblicazione: (2024)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
di: Wang, Zhilin, et al.
Pubblicazione: (2025)

If LLMs Have Human-Like Attributes, Then So Does Age of Empires II
di: de Wynter, Adrian
Pubblicazione: (2026)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
di: Xiao, Chaojun, et al.
Pubblicazione: (2024)

Retrieval Head Mechanistically Explains Long-Context Factuality
di: Wu, Wenhao, et al.
Pubblicazione: (2024)

Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning
di: Alizadeh, Meysam, et al.
Pubblicazione: (2023)

UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
di: Xiong, Yizhe, et al.
Pubblicazione: (2025)

Evaluating and Aligning Human Economic Risk Preferences in LLMs
di: Liu, Jiaxin, et al.
Pubblicazione: (2025)

Re:Form -- Reducing Human Priors in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
di: Yan, Chuanhao, et al.
Pubblicazione: (2025)

Fearful Falcons and Angry Llamas: Emotion Category Annotations of Arguments by Humans and LLMs
di: Greschner, Lynn, et al.
Pubblicazione: (2024)

Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement
di: Lu, Junyu, et al.
Pubblicazione: (2025)

Sparse Neurons Carry Strong Signals of Question Ambiguity in LLMs
di: Zhang, Zhuoxuan, et al.
Pubblicazione: (2025)

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
di: Nguyen, Truong, et al.
Pubblicazione: (2026)

Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning
di: Wang, Shanyong, et al.
Pubblicazione: (2026)