Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.01235 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914886035963904 |
|---|---|
| author | Aleo, P. D. Engel, A. W. Narayan, G. Angus, C. R. Malanchev, K. Auchettl, K. Baldassare, V. F. Berres, A. de Boer, T. J. L. Boyd, B. M. Chambers, K. C. Davis, K. W. Esquivel, N. Farias, D. Foley, R. J. Gagliano, A. Gall, C. Gao, H. Gomez, S. Grayling, M. Jones, D. O. Lin, C. -C. Magnier, E. A. Mandel, K. S. Matheson, T. Raimundo, S. I. Shah, V. G. Soraisam, M. D. de Soto, K. M. Vicencio, S. Villar, V. A. Wainscoat, R. J. |
| author_facet | Aleo, P. D. Engel, A. W. Narayan, G. Angus, C. R. Malanchev, K. Auchettl, K. Baldassare, V. F. Berres, A. de Boer, T. J. L. Boyd, B. M. Chambers, K. C. Davis, K. W. Esquivel, N. Farias, D. Foley, R. J. Gagliano, A. Gall, C. Gao, H. Gomez, S. Grayling, M. Jones, D. O. Lin, C. -C. Magnier, E. A. Mandel, K. S. Matheson, T. Raimundo, S. I. Shah, V. G. Soraisam, M. D. de Soto, K. M. Vicencio, S. Villar, V. A. Wainscoat, R. J. |
| contents | We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host-galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host-galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency ($\sim$ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host-galaxy environments. We use analogs for data-driven discovery, characterization, (re-)classification, and imputation in retrospective and real-time searches. To date we have identified $\sim$50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to: SLSNe, TDEs, SNe IIn, SNe IIb, SNe Ia-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018-2021 and absent from public catalogs ($\sim$1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the "needle in the haystack" in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01235 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams Aleo, P. D. Engel, A. W. Narayan, G. Angus, C. R. Malanchev, K. Auchettl, K. Baldassare, V. F. Berres, A. de Boer, T. J. L. Boyd, B. M. Chambers, K. C. Davis, K. W. Esquivel, N. Farias, D. Foley, R. J. Gagliano, A. Gall, C. Gao, H. Gomez, S. Grayling, M. Jones, D. O. Lin, C. -C. Magnier, E. A. Mandel, K. S. Matheson, T. Raimundo, S. I. Shah, V. G. Soraisam, M. D. de Soto, K. M. Vicencio, S. Villar, V. A. Wainscoat, R. J. High Energy Astrophysical Phenomena Instrumentation and Methods for Astrophysics We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host-galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host-galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency ($\sim$ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host-galaxy environments. We use analogs for data-driven discovery, characterization, (re-)classification, and imputation in retrospective and real-time searches. To date we have identified $\sim$50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to: SLSNe, TDEs, SNe IIn, SNe IIb, SNe Ia-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018-2021 and absent from public catalogs ($\sim$1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the "needle in the haystack" in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data. |
| title | Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams |
| topic | High Energy Astrophysical Phenomena Instrumentation and Methods for Astrophysics |
| url | https://arxiv.org/abs/2404.01235 |