Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.01980 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912452464082944 |
|---|---|
| author | Hayden, David S. Ye, Mao Garipov, Timur Meyer, Gregory P. Vondrick, Carl Chen, Zhao Chai, Yuning Wolff, Eric Srinivasa, Siddhartha S. |
| author_facet | Hayden, David S. Ye, Mao Garipov, Timur Meyer, Gregory P. Vondrick, Carl Chen, Zhao Chai, Yuning Wolff, Eric Srinivasa, Siddhartha S. |
| contents | It is difficult to anticipate the myriad challenges that a predictive model will encounter once deployed. Common practice entails a reactive, cyclical approach: model deployment, data mining, and retraining. We instead develop a proactive longtail discovery process by imagining additional data during training. In particular, we develop general model-based longtail signals, including a differentiable, single forward pass formulation of epistemic uncertainty that does not impact model parameters or predictive performance but can flag rare or hard inputs. We leverage these signals as guidance to generate additional training data from a latent diffusion model in a process we call Longtail Guidance (LTG). Crucially, we can perform LTG without retraining the diffusion model or the predictive model, and we do not need to expose the predictive model to intermediate diffusion states. Data generated by LTG exhibit semantically meaningful variation, yield significant generalization improvements on numerous image classification benchmarks, and can be analyzed by a VLM to proactively discover, textually explain, and address conceptual gaps in a deployed predictive model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01980 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generative Data Mining with Longtail-Guided Diffusion Hayden, David S. Ye, Mao Garipov, Timur Meyer, Gregory P. Vondrick, Carl Chen, Zhao Chai, Yuning Wolff, Eric Srinivasa, Siddhartha S. Machine Learning Artificial Intelligence It is difficult to anticipate the myriad challenges that a predictive model will encounter once deployed. Common practice entails a reactive, cyclical approach: model deployment, data mining, and retraining. We instead develop a proactive longtail discovery process by imagining additional data during training. In particular, we develop general model-based longtail signals, including a differentiable, single forward pass formulation of epistemic uncertainty that does not impact model parameters or predictive performance but can flag rare or hard inputs. We leverage these signals as guidance to generate additional training data from a latent diffusion model in a process we call Longtail Guidance (LTG). Crucially, we can perform LTG without retraining the diffusion model or the predictive model, and we do not need to expose the predictive model to intermediate diffusion states. Data generated by LTG exhibit semantically meaningful variation, yield significant generalization improvements on numerous image classification benchmarks, and can be analyzed by a VLM to proactively discover, textually explain, and address conceptual gaps in a deployed predictive model. |
| title | Generative Data Mining with Longtail-Guided Diffusion |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.01980 |