Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2412.15251 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913083899772928 |
|---|---|
| author | Liu, Mingchao Sun, Yu Sun, Ruixiao Dong, Xin Shen, Xiang Wang, Hongwei Xiong, Hongyu Song, Yang |
| author_facet | Liu, Mingchao Sun, Yu Sun, Ruixiao Dong, Xin Shen, Xiang Wang, Hongwei Xiong, Hongyu Song, Yang |
| contents | Multimodal large language models (MLLMs) are effective at capturing the semantics of short video content; however, they often fail to attend to the policy-specific details required for reliable content moderation. To address this limitation, we introduce IPS, a novel framework that integrates In-prompt Process Supervision into MLLMs by introducing sequential reasoning over ancillary questions during fine-tuning. IPS consistently outperforms baseline MLLMs on public and proprietary benchmarks. Moreover, replacing human-annotated ancillary labels with MLLM-generated ones results in only marginal performance degradation, demonstrating robustness to noisy supervision and strong scalability with model-generated annotations. These findings establish IPS as a scalable and effective solution for complex multimodal classification in large-scale industrial settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15251 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | IPS: In-Prompt Process Supervision for Short Video Content Moderation Liu, Mingchao Sun, Yu Sun, Ruixiao Dong, Xin Shen, Xiang Wang, Hongwei Xiong, Hongyu Song, Yang Computation and Language Artificial Intelligence Multimodal large language models (MLLMs) are effective at capturing the semantics of short video content; however, they often fail to attend to the policy-specific details required for reliable content moderation. To address this limitation, we introduce IPS, a novel framework that integrates In-prompt Process Supervision into MLLMs by introducing sequential reasoning over ancillary questions during fine-tuning. IPS consistently outperforms baseline MLLMs on public and proprietary benchmarks. Moreover, replacing human-annotated ancillary labels with MLLM-generated ones results in only marginal performance degradation, demonstrating robustness to noisy supervision and strong scalability with model-generated annotations. These findings establish IPS as a scalable and effective solution for complex multimodal classification in large-scale industrial settings. |
| title | IPS: In-Prompt Process Supervision for Short Video Content Moderation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.15251 |