Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910864849764352 |
|---|---|
| author | Yuxuan, Cao Jiayang, Wu Chuen, Alistair Cheong Liang Guanrong, Bryan Shan Jen, Theodore Lee Chong Shen, Sherman Chann Zhi |
| author_facet | Yuxuan, Cao Jiayang, Wu Chuen, Alistair Cheong Liang Guanrong, Bryan Shan Jen, Theodore Lee Chong Shen, Sherman Chann Zhi |
| contents | Traditional online content moderation systems struggle to classify modern multimodal means of communication, such as memes, a highly nuanced and information-dense medium. This task is especially hard in a culturally diverse society like Singapore, where low-resource languages are used and extensive knowledge on local context is needed to interpret online content. We curate a large collection of 112K memes labeled by GPT-4V for fine-tuning a VLM to classify offensive memes in Singapore context. We show the effectiveness of fine-tuned VLMs on our dataset, and propose a pipeline containing OCR, translation and a 7-billion parameter-class VLM. Our solutions reach 80.62% accuracy and 0.8192 AUROC on a held-out test set, and can greatly aid human in moderating online contents. The dataset, code, and model weights have been open-sourced at https://github.com/aliencaocao/vlm-for-memes-aisg. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18101 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models Yuxuan, Cao Jiayang, Wu Chuen, Alistair Cheong Liang Guanrong, Bryan Shan Jen, Theodore Lee Chong Shen, Sherman Chann Zhi Computer Vision and Pattern Recognition Computation and Language Traditional online content moderation systems struggle to classify modern multimodal means of communication, such as memes, a highly nuanced and information-dense medium. This task is especially hard in a culturally diverse society like Singapore, where low-resource languages are used and extensive knowledge on local context is needed to interpret online content. We curate a large collection of 112K memes labeled by GPT-4V for fine-tuning a VLM to classify offensive memes in Singapore context. We show the effectiveness of fine-tuned VLMs on our dataset, and propose a pipeline containing OCR, translation and a 7-billion parameter-class VLM. Our solutions reach 80.62% accuracy and 0.8192 AUROC on a held-out test set, and can greatly aid human in moderating online contents. The dataset, code, and model weights have been open-sourced at https://github.com/aliencaocao/vlm-for-memes-aisg. |
| title | Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2502.18101 |