Refine Neutrino Events Reconstruction with BEiT-3
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916550312722432 |
|---|---|
| author | Li, Chen Cai, Hao Jiang, Xianyang |
| author_facet | Li, Chen Cai, Hao Jiang, Xianyang |
| contents | Neutrino Events Reconstruction has always been crucial for IceCube Neutrino Observatory. In the Kaggle competition "IceCube -- Neutrinos in Deep Ice", many solutions use Transformer. We present ISeeCube, a pure Transformer model based on TorchScale (the backbone of BEiT-3). When having relatively same amount of total trainable parameters, our model outperforms the 2nd place solution. By using TorchScale, the lines of code drop sharply by about 80% and a lot of new methods can be tested by simply adjusting configs. We compared two fundamental models for predictions on a continuous space, regression and classification, trained with MSE Loss and CE Loss respectively. We also propose a new metric, overlap ratio, to evaluate the performance of the model. Since the model is simple enough, it has the potential to be used for more purposes such as energy reconstruction, and many new methods such as combining it with GraphNeT can be tested more easily. The code and pretrained models are available at https://github.com/chenlinear/ISeeCube |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_13285 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Refine Neutrino Events Reconstruction with BEiT-3 Li, Chen Cai, Hao Jiang, Xianyang Instrumentation and Methods for Astrophysics High Energy Astrophysical Phenomena High Energy Physics - Experiment High Energy Physics - Phenomenology Neutrino Events Reconstruction has always been crucial for IceCube Neutrino Observatory. In the Kaggle competition "IceCube -- Neutrinos in Deep Ice", many solutions use Transformer. We present ISeeCube, a pure Transformer model based on TorchScale (the backbone of BEiT-3). When having relatively same amount of total trainable parameters, our model outperforms the 2nd place solution. By using TorchScale, the lines of code drop sharply by about 80% and a lot of new methods can be tested by simply adjusting configs. We compared two fundamental models for predictions on a continuous space, regression and classification, trained with MSE Loss and CE Loss respectively. We also propose a new metric, overlap ratio, to evaluate the performance of the model. Since the model is simple enough, it has the potential to be used for more purposes such as energy reconstruction, and many new methods such as combining it with GraphNeT can be tested more easily. The code and pretrained models are available at https://github.com/chenlinear/ISeeCube |
| title | Refine Neutrino Events Reconstruction with BEiT-3 |
| topic | Instrumentation and Methods for Astrophysics High Energy Astrophysical Phenomena High Energy Physics - Experiment High Energy Physics - Phenomenology |
| url | https://arxiv.org/abs/2308.13285 |