Cover Image

Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Gallego, Víctor
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Spontaneous Reward Hacking in Iterative Self-Refinement
by: Pan, Jane, et al.
Published: (2024)

MetaSC: Test-Time Safety Specification Optimization for Language Models
by: Gallego, Víctor
Published: (2025)

Reward Shaping to Mitigate Reward Hacking in RLHF
by: Fu, Jiayi, et al.
Published: (2025)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
by: Chen, Lichang, et al.
Published: (2024)

Temper and Tilt Lead to SLOP: Reward Hacking Mitigation with Inference-Time Alignment
by: Wang, Ye, et al.
Published: (2026)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
by: Ono, Shinnosuke, et al.
Published: (2026)

Feedback Loops With Language Models Drive In-Context Reward Hacking
by: Pan, Alexander, et al.
Published: (2024)

Discovering Agentic Safety Specifications from 1-Bit Danger Signals
by: Gallego, Víctor
Published: (2026)

CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute
by: Jin, Chen, et al.
Published: (2026)

Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning
by: Turpin, Miles, et al.
Published: (2025)

Merging Improves Self-Critique Against Jailbreak Attacks
by: Gallego, Victor
Published: (2024)

Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
by: Wang, Xinpeng, et al.
Published: (2025)

Monitoring Emergent Reward Hacking During Generation via Internal Activations
by: Wilhelm, Patrick, et al.
Published: (2026)

SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling
by: Chen, Jiefeng, et al.
Published: (2025)

Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards
by: Ackermann, Johannes, et al.
Published: (2026)

SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
by: Zhao, Bingchen, et al.
Published: (2026)

Countdown-Code: A Testbed for Studying The Emergence and Generalization of Reward Hacking in RLVR
by: Khalifa, Muhammad, et al.
Published: (2026)

Logical Reasoning with Outcome Reward Models for Test-Time Scaling
by: Thatikonda, Ramya Keerthy, et al.
Published: (2025)

Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences
by: Karlekar, Sweta, et al.
Published: (2026)

Configurable Preference Tuning with Rubric-Guided Synthetic Data
by: Gallego, Víctor
Published: (2025)

Configurable Safety Tuning of Language Models with Synthetic Preference Data
by: Gallego, Victor
Published: (2024)

Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning
by: Zhou, Yuxuan, et al.
Published: (2025)

Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation
by: Lee, Nakyung, et al.
Published: (2025)

Intent-Aware Self-Correction for Mitigating Social Biases in Large Language Models
by: Anantaprayoon, Panatchakorn, et al.
Published: (2025)

Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching
by: Miles, Roy, et al.
Published: (2026)

SCIR: A Self-Correcting Iterative Refinement Framework for Enhanced Information Extraction Based on Schema
by: Fang, Yushen, et al.
Published: (2025)

ATLAS: Learning to Optimally Memorize the Context at Test Time
by: Behrouz, Ali, et al.
Published: (2025)

Calibration Collapse Under Sycophancy Fine-Tuning: How Reward Hacking Breaks Uncertainty Quantification in LLMs
by: Sahoo, Subramanyam
Published: (2026)

Improving Fairness in LLMs Through Testing-Time Adversaries
by: Gregio, Isabela Pereira, et al.
Published: (2025)

A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models
by: Han, Jinyi, et al.
Published: (2025)

Entropy Centroids as Intrinsic Rewards for Test-Time Scaling
by: Zhao, Wenshuo, et al.
Published: (2026)

Self-Rewarding Language Models
by: Yuan, Weizhe, et al.
Published: (2024)

Evaluating Language Model Context Windows: A "Working Memory" Test and Inference-time Correction
by: Dsouza, Amanda, et al.
Published: (2024)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
by: Peng, Hao, et al.
Published: (2025)

EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems
by: He, Yufei, et al.
Published: (2025)

Simple Hack for Transformers against Heavy Long-Text Classification on a Time- and Memory-Limited GPU Service
by: Mutasodirin, Mirza Alim, et al.
Published: (2024)

Self-Evolved Reward Learning for LLMs
by: Huang, Chenghua, et al.
Published: (2024)

Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement
by: Lee, Daeun, et al.
Published: (2024)

A Dual-Directional Context-Aware Test-Time Learning for Text Classification
by: Xu, Dong, et al.
Published: (2025)

Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
by: Xu, Wenda, et al.
Published: (2024)