Cover Image

FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hayes, Kevin David, Goldblum, Micah, Sehwag, Vikash, Somepalli, Gowthami, Panda, Ashwinee, Goldstein, Tom
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Measuring Style Similarity in Diffusion Models
by: Somepalli, Gowthami, et al.
Published: (2024)

Multi-Token Prediction via Self-Distillation
by: Kirchenbauer, John, et al.
Published: (2026)

Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models
by: Shirkavand, Reza, et al.
Published: (2024)

Identifying and Evaluating Inactive Heads in Pretrained LLMs
by: Sandoval-Segura, Pedro, et al.
Published: (2025)

ARGUS: Hallucination and Omission Evaluation in Video-LLMs
by: Rawal, Ruchit, et al.
Published: (2025)

Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models
by: Jain, Neel, et al.
Published: (2024)

Gemstones: A Model Suite for Multi-Faceted Scaling Laws
by: McLeish, Sean, et al.
Published: (2025)

A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization
by: Panda, Ashwinee, et al.
Published: (2022)

SolidMark: Evaluating Image Memorization in Generative Models
by: Kriplani, Nicky, et al.
Published: (2025)

LMD3: Language Model Data Density Dependence
by: Kirchenbauer, John, et al.
Published: (2024)

Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking
by: Feuer, Benjamin, et al.
Published: (2024)

CinePile: A Long Video Question Answering Dataset and Benchmark
by: Rawal, Ruchit, et al.
Published: (2024)

LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation
by: Zhang, Juzheng, et al.
Published: (2025)

Private Fine-tuning of Large Language Models with Zeroth-order Optimization
by: Tang, Xinyu, et al.
Published: (2024)

PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach
by: Sehwag, Udari Madhushani, et al.
Published: (2025)

Analysis of Attention in Video Diffusion Transformers
by: Wen, Yuxin, et al.
Published: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
by: Madan, Vivan, et al.
Published: (2026)

From Pixels to Prose: A Large Dataset of Dense Image Captions
by: Singla, Vasu, et al.
Published: (2024)

Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
by: Li, Ang, et al.
Published: (2025)

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
by: Hans, Abhimanyu, et al.
Published: (2024)

On the Reliability of Watermarks for Large Language Models
by: Kirchenbauer, John, et al.
Published: (2023)

Self-Comparison for Dataset-Level Membership Inference in Large (Vision-)Language Models
by: Ren, Jie, et al.
Published: (2024)

vTune: Verifiable Fine-Tuning for LLMs Through Backdooring
by: Zhang, Eva, et al.
Published: (2024)

Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation
by: Lee, Seongyun, et al.
Published: (2024)

Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
by: McLeish, Sean, et al.
Published: (2025)

Evaluation Awareness Scales Predictably in Open-Weights Large Language Models
by: Chaudhary, Maheep, et al.
Published: (2025)

Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions
by: Pal, Arka, et al.
Published: (2025)

How to Trace Latent Generative Model Generated Images without Artificial Watermark?
by: Wang, Zhenting, et al.
Published: (2024)

Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
by: Li, Ang, et al.
Published: (2025)

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
by: Panda, Ashwinee, et al.
Published: (2025)

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
by: Hans, Abhimanyu, et al.
Published: (2024)

Modeling and Predicting Multi-Turn Answer Instability in Large Language Models
by: He, Jiahang, et al.
Published: (2025)

Evaluating and Mitigating IP Infringement in Visual Generative AI
by: Wang, Zhenting, et al.
Published: (2024)

Privacy Auditing of Large Language Models
by: Panda, Ashwinee, et al.
Published: (2025)

CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI
by: Cheng, Siyuan, et al.
Published: (2025)

Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
by: Marek, Martin, et al.
Published: (2025)

Discovering Failure Modes in Vision-Language Models using RL
by: Jain, Kanishk, et al.
Published: (2026)

Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
by: Lotfi, Sanae, et al.
Published: (2024)

Teach LLMs to Phish: Stealing Private Information from Language Models
by: Panda, Ashwinee, et al.
Published: (2024)

A Simple Baseline for Predicting Events with Auto-Regressive Tabular Transformers
by: Stein, Alex, et al.
Published: (2024)