Cover Image

Compared to What? Baselines and Metrics for Counterfactual Prompting

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Zihao, Levy, Mosh, Goldberg, Yoav, Wallace, Byron C.
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
by: Levy, Mosh, et al.
Published: (2024)

Humans Perceive Wrong Narratives from AI Reasoning Texts
by: Levy, Mosh, et al.
Published: (2025)

Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature
by: Katz, Uri, et al.
Published: (2024)

A Practical Method for Generating String Counterfactuals
by: Avitan, Matan, et al.
Published: (2024)

State over Tokens: Characterizing the Role of Reasoning Tokens
by: Levy, Mosh, et al.
Published: (2025)

Can SAEs reveal and mitigate racial biases of LLMs in healthcare?
by: Ahsan, Hiba, et al.
Published: (2025)

PRISM: PRIor from corpus Statistics for topic Modeling
by: Ishon, Tal, et al.
Published: (2026)

Log-linear Guardedness and its Implications
by: Ravfogel, Shauli, et al.
Published: (2022)

No Mean Feat: Simple, Strong Baselines for Context Compression
by: Feldman, Yair, et al.
Published: (2025)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
by: Ben-Zaken, Elad, et al.
Published: (2021)

Kernelized Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

Linear Adversarial Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

Diversity Over Quantity: A Lesson From Few Shot Relation Classification
by: Cohen, Amir DN, et al.
Published: (2024)

Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
by: Roy, Debjyoti Saha, et al.
Published: (2024)

Future Lens: Anticipating Subsequent Tokens from a Single Hidden State
by: Pal, Koyena, et al.
Published: (2023)

Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains
by: Ramprasad, Sanjana, et al.
Published: (2024)

Do Activation Verbalization Methods Convey Privileged Information?
by: Li, Millicent, et al.
Published: (2025)

GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
by: Krishna, Kundan, et al.
Published: (2024)

Description-Based Text Similarity
by: Ravfogel, Shauli, et al.
Published: (2023)

What Evidence Do Language Models Find Convincing?
by: Wan, Alexander, et al.
Published: (2024)

Prompt-Counterfactual Explanations for Generative AI System Behavior
by: Goethals, Sofie, et al.
Published: (2026)

Function Vectors in Large Language Models
by: Todd, Eric, et al.
Published: (2023)

Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
by: Feucht, Sheridan, et al.
Published: (2024)

PIER: A Novel Metric for Evaluating What Matters in Code-Switching
by: Ugan, Enes Yavuz, et al.
Published: (2025)

Estimating LLM Consistency: A User Baseline vs Surrogate Metrics
by: Wu, Xiaoyuan, et al.
Published: (2025)

Counterfactual Generation with Identifiability Guarantees
by: Yan, Hanqi, et al.
Published: (2024)

Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
by: Cohen, Eilam, et al.
Published: (2026)

Stronger Random Baselines for In-Context Learning
by: Yauney, Gregory, et al.
Published: (2024)

On-Policy RL with Optimal Reward Baseline
by: Hao, Yaru, et al.
Published: (2025)

Aligning (Medical) LLMs for (Counterfactual) Fairness
by: Poulain, Raphael, et al.
Published: (2024)

Why Don't Prompt-Based Fairness Metrics Correlate?
by: Zayed, Abdelrahman, et al.
Published: (2024)

Transpose Attack: Stealing Datasets with Bidirectional Training
by: Amit, Guy, et al.
Published: (2023)

What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models
by: Zhang, Letian, et al.
Published: (2023)

PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models
by: Zhao, Chenzhuo, et al.
Published: (2025)

Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
by: Zhang, Jingyang, et al.
Published: (2024)

Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages
by: Li, Zihao, et al.
Published: (2024)

Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation
by: Ramprasad, Sanjana, et al.
Published: (2024)

iFlip: Iterative Feedback-driven Counterfactual Example Refinement
by: Wang, Yilong, et al.
Published: (2026)

Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient
by: Shi, Zeru, et al.
Published: (2024)

Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
by: Leyton-Brown, Kevin, et al.
Published: (2024)