Cover Image

Compositional preference models for aligning LMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Go, Dongyoung, Korbak, Tomasz, Kruszewski, Germán, Rozen, Jos, Dymetman, Marc
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity
by: Kruszewski, Germán, et al.
Published: (2025)

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data
by: Thonet, Thibaut, et al.
Published: (2025)

Aligning language models with human preferences
by: Korbak, Tomasz
Published: (2024)

Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
by: Kuciński, Łukasz, et al.
Published: (2021)

ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models
by: Thonet, Thibaut, et al.
Published: (2024)

Guaranteed Generation from Large Language Models
by: Kim, Minbeom, et al.
Published: (2024)

LIRE: listwise reward enhancement for preference alignment
by: Zhu, Mingye, et al.
Published: (2024)

Inference time LLM alignment in single and multidomain preference spectrum
by: Shahriar, Sadat, et al.
Published: (2024)

Binary Rewards and Reinforcement Learning: Fundamental Challenges
by: Dymetman, Marc
Published: (2026)

Enabling Approximate Joint Sampling in Diffusion LMs
by: Bansal, Parikshit, et al.
Published: (2025)

Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
by: Ahmed, Kareem, et al.
Published: (2026)

Training Bilingual LMs with Data Constraints in the Targeted Language
by: Seto, Skyler, et al.
Published: (2024)

Exponential families from a single KL identity
by: Dymetman, Marc
Published: (2026)

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
by: Qin, Guanghui, et al.
Published: (2021)

Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
by: Liu, Hengchang, et al.
Published: (2026)

Reflect-RL: Two-Player Online RL Fine-Tuning for LMs
by: Zhou, Runlong, et al.
Published: (2024)

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
by: Berglund, Lukas, et al.
Published: (2023)

Training Language Models with Language Feedback at Scale
by: Scheurer, Jérémy, et al.
Published: (2023)

MUStReason: A Benchmark for Diagnosing Pragmatic Reasoning in Video-LMs for Multimodal Sarcasm Detection
by: Saha, Anisha, et al.
Published: (2025)

Learning to Correct for QA Reasoning with Black-box LLMs
by: Kim, Jaehyung, et al.
Published: (2024)

Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
by: Damani, Mehul, et al.
Published: (2025)

Grounded Token Initialization for New Vocabulary in LMs for Generative Recommendation
by: Chen, Daiwei, et al.
Published: (2026)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
by: Rawat, Ankit Singh, et al.
Published: (2024)

SoftLMs: Efficient Adaptive Low-Rank Approximation of Language Models using Soft-Thresholding Mechanism
by: Bhatnagar, Priyansh, et al.
Published: (2024)

When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
by: Jeong, Soyeong, et al.
Published: (2025)

Addressing the Ecological Fallacy in Larger LMs with Human Context
by: Soni, Nikita, et al.
Published: (2026)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Learning from Synthetic Data Improves Multi-hop Reasoning
by: Kabra, Anmol, et al.
Published: (2026)

Enhancing TinyBERT for Financial Sentiment Analysis Using GPT-Augmented FinBERT Distillation
by: Thomas, Graison Jos
Published: (2024)

InfAlign: Inference-aware language model alignment
by: Balashankar, Ananth, et al.
Published: (2024)

Anatomical Heterogeneity in Transformer Language Models
by: Wietrzykowski, Tomasz
Published: (2026)

ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization
by: Ahuja, Riyaz, et al.
Published: (2026)

The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious
by: Chua, James, et al.
Published: (2026)

Improving Code Generation by Training with Natural Language Feedback
by: Chen, Angelica, et al.
Published: (2023)

Pre-training Limited Memory Language Models with Internal and External Knowledge
by: Zhao, Linxi, et al.
Published: (2025)

OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
by: Asai, Akari, et al.
Published: (2024)

$\textbf{PLUM}$: Improving Code LMs with Execution-Guided On-Policy Preference Learning Driven By Synthetic Test Cases
by: Zhang, Dylan, et al.
Published: (2024)

SentinelLMs: Encrypted Input Adaptation and Fine-tuning of Language Models for Private and Secure Inference
by: Mishra, Abhijit, et al.
Published: (2023)

NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness
by: Singhal, Manav, et al.
Published: (2024)

Are aligned neural networks adversarially aligned?
by: Carlini, Nicholas, et al.
Published: (2023)