Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.07738 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918238179295232 |
|---|---|
| author | Zhang, Dengjia Weng, Charles Guerrerio, Katherine Lu, Yi Murray, Kenton Martin, Alexander Kriz, Reno Van Durme, Benjamin |
| author_facet | Zhang, Dengjia Weng, Charles Guerrerio, Katherine Lu, Yi Murray, Kenton Martin, Alexander Kriz, Reno Van Durme, Benjamin |
| contents | The HLTCOE Evaluation team participated in TREC VQA's Answer Generation (AG) task, for which we developed a listwise learning framework that aims to improve semantic precision and ranking consistency in answer generation. Given a video-question pair, a base multimodal model first generates multiple candidate answers, which are then reranked using a model trained with a novel Masked Pointer Cross-Entropy Loss with Rank Weights. This objective integrates pointer-based candidate selection, rank-dependent weighting, and masked cross-entropy under vocabulary restriction, enabling stable and interpretable listwise optimization. By bridging generative modeling with discriminative ranking, our method produces coherent, fine-grained answer lists. Experiments reveal consistent gains in accuracy and ranking stability, especially for questions requiring temporal reasoning and semantic disambiguation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_07738 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HLTCOE Evaluation Team at TREC 2025: VQA Track Zhang, Dengjia Weng, Charles Guerrerio, Katherine Lu, Yi Murray, Kenton Martin, Alexander Kriz, Reno Van Durme, Benjamin Computer Vision and Pattern Recognition The HLTCOE Evaluation team participated in TREC VQA's Answer Generation (AG) task, for which we developed a listwise learning framework that aims to improve semantic precision and ranking consistency in answer generation. Given a video-question pair, a base multimodal model first generates multiple candidate answers, which are then reranked using a model trained with a novel Masked Pointer Cross-Entropy Loss with Rank Weights. This objective integrates pointer-based candidate selection, rank-dependent weighting, and masked cross-entropy under vocabulary restriction, enabling stable and interpretable listwise optimization. By bridging generative modeling with discriminative ranking, our method produces coherent, fine-grained answer lists. Experiments reveal consistent gains in accuracy and ranking stability, especially for questions requiring temporal reasoning and semantic disambiguation. |
| title | HLTCOE Evaluation Team at TREC 2025: VQA Track |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.07738 |