Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.13216 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916329303310336 |
|---|---|
| author | Vuong, Trinh T. L. Bui, Doanh C. Kwak, Jin Tae |
| author_facet | Vuong, Trinh T. L. Bui, Doanh C. Kwak, Jin Tae |
| contents | In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network's core for enhanced performance. Our solutions achieve the $2^{nd}$ rank in action recognition and anticipation tasks and $1^{st}$ rank in the VQA task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13216 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | QuIIL at T3 challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View Vuong, Trinh T. L. Bui, Doanh C. Kwak, Jin Tae Computer Vision and Pattern Recognition In this paper, we present our solutions for a spectrum of automation tasks in life-saving intervention procedures within the Trauma THOMPSON (T3) Challenge, encompassing action recognition, action anticipation, and Visual Question Answering (VQA). For action recognition and anticipation, we propose a pre-processing strategy that samples and stitches multiple inputs into a single image and then incorporates momentum- and attention-based knowledge distillation to improve the performance of the two tasks. For training, we present an action dictionary-guided design, which consistently yields the most favorable results across our experiments. In the realm of VQA, we leverage object-level features and deploy co-attention networks to train both object and question features. Notably, we introduce a novel frame-question cross-attention mechanism at the network's core for enhanced performance. Our solutions achieve the $2^{nd}$ rank in action recognition and anticipation tasks and $1^{st}$ rank in the VQA task. |
| title | QuIIL at T3 challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.13216 |