notesum.ai

Published at November 19

Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes

cs.AI

cs.CL

cs.CV

Released Date: November 19, 2024

Authors: Rahul Garg¹, Trilok Padhi², Hemang Jain, Ugur Kursuncu², Ponnurangam Kumaraguru

Aff.: ¹IIIT Hyderabad, Hyderabad, India; ²Georgia State University, Atlanta, GA, USA

Arxiv: http://arxiv.org/abs/2411.12174v1

Refer to caption

Framework	Knowledge	Accuracy	F1	Precision	Recall	AUC
MMBT	-	68.80	67.61	71.25	68.44	77.84
MMBT	Triples	69.40	68.04	72.46	69.02	78.28
MMBT	Summarized Triples	69.60	68.96	70.83	69.34	78.03
CLIP-Text^*	-	55.30	46.63	70.09	15.31	65.75
CLIP-Image^*	-	62.20	58.98	74.35	34.89	73.42
CLIP^*	-	59.50	53.88	76.40	25.10	72.14
CLIP-Text	-	59.90	56.87	68.16	34.08	68.01
CLIP-Image	-	55.70	53.55	57.97	34.90	60.78
CLIP	-	60.30	57.10	69.62	33.67	68.30
LLaVA⁺	-	57.00	52.43	60.00	56.40	-
PromptHate	-	76.10	75.76	77.30	76.10	84.21
HateClipper	-	76.07	73.00	78.00	56.00	85.14
KID-VLM	Hop 1 & Roberta	78.20 (+2.13)	78.12 (+5.12)	80.36(+2.36)	73.47 (+17.47)	86.02 (+0.88)
KID-VLM	Hop 2 & Roberta	76.30 (+0.23)	76.07 (+3.07)	76.94 (-1.06)	76.14 (+20.14)	86.11 (+0.97)
KID-VLM	Hop 1 & MiniLM	78.70 (+2.63)	78.63 (+5.63)	80.57 (+2.57)	74.49 (+18.49)	86.11 (+0.97)
KID-VLM	Hop 2 & MiniLM	76.30(+0.23)	75.95 (+2.95)	82.52 (+4.52)	65.51 (+9.51)	85.48 (+0.34)