Cover Image

Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference Labels

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fränken, Jan-Philipp, Zelikman, Eric, Rafailov, Rafael, Gandhi, Kanishk, Gerstenberg, Tobias, Goodman, Noah D.
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Models
by: Fränken, Jan-Philipp, et al.
Published: (2024)

STaR-GATE: Teaching Language Models to Ask Clarifying Questions
by: Andukuri, Chinmaya, et al.
Published: (2024)

Human-like Affective Cognition in Foundation Models
by: Gandhi, Kanishk, et al.
Published: (2024)

Non-literal Understanding of Number Words by Language Models
by: Tsvilodub, Polina, et al.
Published: (2025)

Learning to Simulate Human Dialogue
by: Gandhi, Kanishk, et al.
Published: (2026)

PERSONA: A Reproducible Testbed for Pluralistic Alignment
by: Castricato, Louis, et al.
Published: (2024)

Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
by: Gandhi, Kanishk, et al.
Published: (2025)

Endless Terminals: Scaling RL Environments for Terminal Agents
by: Gandhi, Kanishk, et al.
Published: (2026)

Psychometric Alignment: Capturing Human Knowledge Distributions via Language Models
by: He-Yueya, Joy, et al.
Published: (2024)

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
by: Xiang, Violet, et al.
Published: (2025)

Hypothesis Search: Inductive Reasoning with Language Models
by: Wang, Ruocheng, et al.
Published: (2023)

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
by: Zelikman, Eric, et al.
Published: (2024)

Stream of Search (SoS): Learning to Search in Language
by: Gandhi, Kanishk, et al.
Published: (2024)

An Exploration of Self-Supervised Mutual Information Alignment for Multi-Task Settings
by: Govande, Soham V.
Published: (2024)

Scaling up the think-aloud method
by: Wurgaft, Daniel, et al.
Published: (2025)

Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
by: Zelikman, Eric, et al.
Published: (2023)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
by: Albalak, Alon, et al.
Published: (2025)

Self-Supervised Visual Preference Alignment
by: Zhu, Ke, et al.
Published: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
by: Zhou, Yiyang, et al.
Published: (2024)

Bifocal Attention: Harmonizing Geometric and Spectral Positional Embeddings for Algorithmic Generalization
by: Awadhiya, Kanishk
Published: (2026)

ORPO: Monolithic Preference Optimization without Reference Model
by: Hong, Jiwoo, et al.
Published: (2024)

Beyond Preferences: Learning Alignment Principles Grounded in Human Reasons and Values
by: Bell, Henry, et al.
Published: (2026)

On-the-fly Preference Alignment via Principle-Guided Decoding
by: Zhu, Mingye, et al.
Published: (2025)

Learning to Compress Prompts with Gist Tokens
by: Mu, Jesse, et al.
Published: (2023)

Effective Explanations Support Planning Under Uncertainty
by: Zhou, Hanqi, et al.
Published: (2026)

Bayesian Preference Elicitation with Language Models
by: Handa, Kunal, et al.
Published: (2024)

Intrinsic Mutual Information as a Modulator for Preference Optimization
by: Liao, Peng, et al.
Published: (2026)

Self-Judge: Selective Instruction Following with Alignment Self-Evaluation
by: Ye, Hai, et al.
Published: (2024)

Crossing Domains without Labels: Distant Supervision for Term Extraction
by: Senger, Elena, et al.
Published: (2025)

From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models
by: Mishra, Shubhra, et al.
Published: (2024)

Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models
by: Zhang, Christine, et al.
Published: (2026)

STENCIL: Submodular Mutual Information Based Weak Supervision for Cold-Start Active Learning
by: Beck, Nathan, et al.
Published: (2024)

SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters
by: Xiao, Teng, et al.
Published: (2025)

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
by: Rafailov, Rafael, et al.
Published: (2024)

CARMA: Enhanced Compositionality in LLMs via Advanced Regularisation and Mutual Information Alignment
by: Aljaafari, Nura, et al.
Published: (2025)

The Nature and Scope of Student Search Strategies in Using a Web Derived Corpus for Writing
by: Franken, Margaret
Published: (2014)

Supervised In-Context Fine-Tuning for Generative Sequence Labeling
by: Dukić, David, et al.
Published: (2025)

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization
by: Cui, Chaoqun, et al.
Published: (2025)