Photorealistic Inpainting for Perturbation-based Explanations in Ecological Monitoring
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911228984557568 |
|---|---|
| author | Aghakishiyeva, Günel Zhou, Jiayi Arya, Saagar Dale, Julian Poling, James David Houliston, Holly R. Womble, Jamie N. Larsen, Gregory D. Johnston, David W. Bent, Brinnae |
| author_facet | Aghakishiyeva, Günel Zhou, Jiayi Arya, Saagar Dale, Julian Poling, James David Houliston, Holly R. Womble, Jamie N. Larsen, Gregory D. Johnston, David W. Bent, Brinnae |
| contents | Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03317 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Photorealistic Inpainting for Perturbation-based Explanations in Ecological Monitoring Aghakishiyeva, Günel Zhou, Jiayi Arya, Saagar Dale, Julian Poling, James David Houliston, Holly R. Womble, Jamie N. Larsen, Gregory D. Johnston, David W. Bent, Brinnae Computer Vision and Pattern Recognition Artificial Intelligence Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology. |
| title | Photorealistic Inpainting for Perturbation-based Explanations in Ecological Monitoring |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2510.03317 |