Cover Image

Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Laban, Philippe, Murakhovs'ka, Lidiya, Xiong, Caiming, Wu, Chien-Sheng
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
by: Laban, Philippe, et al.
Published: (2024)

Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage
by: Xie, Kaige, et al.
Published: (2024)

SummExecEdit: A Factual Consistency Benchmark in Summarization with Executable Edits
by: Thorat, Onkar, et al.
Published: (2024)

Shared Imagination: LLMs Hallucinate Alike
by: Zhou, Yilun, et al.
Published: (2024)

Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits
by: Chakrabarty, Tuhin, et al.
Published: (2024)

LLMs Corrupt Your Documents When You Delegate
by: Laban, Philippe, et al.
Published: (2026)

Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
by: Hejabi, Parsa, et al.
Published: (2025)

AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation
by: Chakrabarty, Tuhin, et al.
Published: (2025)

CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments
by: Huang, Kung-Hsiang, et al.
Published: (2024)

Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding
by: Huang, Kung-Hsiang, et al.
Published: (2025)

Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
by: Huang, Kung-Hsiang, et al.
Published: (2023)

Flipping the Dialogue: Training and Evaluating User Language Models
by: Naous, Tarek, et al.
Published: (2025)

FlipFlop: A Static Analysis-based Energy Optimization Framework for GPU Kernels
by: Rajput, Saurabhsingh, et al.
Published: (2026)

Agentic Confidence Calibration
by: Zhang, Jiaxin, et al.
Published: (2026)

BingoGuard: LLM Content Moderation Tools with Risk Levels
by: Yin, Fan, et al.
Published: (2025)

ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement
by: Peng, Xiangyu, et al.
Published: (2024)

Unanswerability Evaluation for Retrieval Augmented Generation
by: Peng, Xiangyu, et al.
Published: (2024)

Art or Artifice? Large Language Models and the False Promise of Creativity
by: Chakrabarty, Tuhin, et al.
Published: (2023)

Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
by: Xiang, Yanzheng, et al.
Published: (2026)

MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
by: Tang, Liyan, et al.
Published: (2024)

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness
by: Huang, Kung-Hsiang, et al.
Published: (2025)

Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses
by: Venkit, Pranav Narayanan, et al.
Published: (2024)

LLMs Get Lost In Multi-Turn Conversation
by: Laban, Philippe, et al.
Published: (2025)

The UNDO Flip-Flop: A Controlled Probe for Reversible Semantic State Management in State Space Model
by: Zhou, Hongxu
Published: (2026)

DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence
by: Venkit, Pranav Narayanan, et al.
Published: (2025)

UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG
by: Peng, Xiangyu, et al.
Published: (2025)

Agentic Uncertainty Quantification
by: Zhang, Jiaxin, et al.
Published: (2026)

Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs
by: Alghallabi, Wafa, et al.
Published: (2025)

Prompt Leakage effect and defense strategies for multi-turn LLM interactions
by: Agarwal, Divyansh, et al.
Published: (2024)

Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents
by: Choubey, Prafulla Kumar, et al.
Published: (2025)

Supervised Optimism Correction: Be Confident When LLMs Are Sure
by: Zhang, Junjie, et al.
Published: (2025)

Mic Drop or Data Flop? Evaluating the Fitness for Purpose of AI Voice Interviewers for Data Collection within Quantitative & Qualitative Research Contexts
by: Tirumala, Shreyas, et al.
Published: (2025)

Benchmarking Deep Search over Heterogeneous Enterprise Data
by: Choubey, Prafulla Kumar, et al.
Published: (2025)

Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains
by: Xu, Austin, et al.
Published: (2025)

Are You Sure You're Positive? Consolidating Chain-of-Thought Agents with Uncertainty Quantification for Aspect-Category Sentiment Analysis
by: Ventirozos, Filippos, et al.
Published: (2025)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
by: Devine, Peter
Published: (2024)

Convomem Benchmark: Why Your First 150 Conversations Don't Need RAG
by: Pakhomov, Egor, et al.
Published: (2025)

SiReRAG: Indexing Similar and Related Information for Multihop Reasoning
by: Zhang, Nan, et al.
Published: (2024)

Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
by: Zhou, Yixiao, et al.
Published: (2025)

FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability
by: Xia, Congying, et al.
Published: (2024)