notesum.ai

Published at November 26

Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation

cs.CV

Released Date: November 26, 2024

Authors: Chanyoung Kim¹, Dayun Ju¹, Woojung Han¹, Ming-Hsuan Yang², Seong Jae Hwang¹

Aff.: ¹Yonsei University; ²University of California, Merced

Arxiv: http://arxiv.org/abs/2411.17150v1

Refer to caption

Model		Supporting Dataset	Extra-Training	Fair	V21	PC60	C-Obj	V20	PC59	C-Stf	City	ADE	Avg.
GroupViT [49]	CVPR’22	CC12M+RedCaps	✓	✗	50.4	18.7	27.5	79.7	23.4	15.3	11.1	9.2	29.4
TCL [9]	CVPR’23	CC3M+CC12M	✓	✗	55.0	30.4	31.6	83.2	33.9	22.4	24.0	17.1	37.2
CoDe [46]	CVPR’24	CC3M+RedCaps	✓	✗	57.5	30.5	32.3	-	-	23.9	28.9	17.7	-
CLIP-DINOiser [47]	ECCV’24	ImageNet1k	✓	✗	62.1	32.4	34.8	80.9	35.9	24.6	31.7	20.0	40.3
ReCo [40]	NeurIPS’22	ImageNet1k	✗	✗	25.1	19.9	15.7	57.7	22.3	14.8	21.6	11.2	23.5
FOSSIL [3]	WACV’24	COCO Captions	✗	✗	-	-	-	-	35.8	24.8	23.2	18.8	-
FreeDa [4]	CVPR’24	COCO Captions	✗	✗	-	-	-	85.6	43.1	27.8	36.7	22.4	-
CLIP [34]	ICML’21	✗	✗	✓	18.6	7.8	6.5	49.1	11.2	7.2	6.7	3.2	13.8
MaskCLIP [53]	ECCV’22	✗	✗	✓	38.3	23.6	20.6	74.9	26.4	16.4	12.6	9.8	27.9
GEM [6]	CVPR’24	✗	✗	✓	46.2	-	-	-	32.6	-	-	15.7	-
CaR [42]	CVPR’24	✗	✗	✓	48.6	13.6	15.4	73.7	18.4	-	-	5.4	-
PnP-OVSS [42]	CVPR’24	✗	✗	✓	-	-	36.2	51.3	28.0	17.9	-	14.2	-
CLIPtrase [39]	ECCV’24	✗	✗	✓	50.9	29.9	43.6	81.0	33.8	22.8	21.3	16.4	32.7
ClearCLIP [24]	ECCV’24	✗	✗	✓	51.8	32.6	33.0	80.9	35.9	23.9	30.0	16.7	38.1
SCLIP [43]	ECCV’24	✗	✗	✓	59.1	30.4	30.5	80.4	34.1	22.4	32.2	16.1	38.2
LaVG [20]	ECCV’24	✗	✗	✓	62.1	31.6	34.2	82.5	34.7	23.2	26.2	15.8	38.8
ProxyCLIP [25]	ECCV’24	✗	✗	✓	59.1	35.2	36.2	78.2	38.8	26.2	38.1	19.6	41.4
NACLIP [16]	WACV’25	✗	✗	✓	58.9	32.2	33.2	79.7	35.2	23.3	35.5	17.4	39.4
CASS		✗	✗	✓	65.8	36.7	37.8	87.8	40.2	26.7	39.4	20.4	44.4