notesum.ai

Published at November 29

Effective Fine-Tuning of Vision-Language Models for Accurate Galaxy Morphology Analysis

cs.CV

astro-ph.GA

cs.AI

cs.LG

Released Date: November 29, 2024

Authors: Ruoqi Wang¹, Haitao Wang², Qiong Luo³

Aff.: ¹HKUST(GZ); ²SYSU; ³HKUST(GZ) & HKUST

Arxiv: http://arxiv.org/pdf/2411.19475v1

Refer to caption

Method	Pretraining Dataset		GalaxyMNIST		Galaxy10
Method	General	Domain	Accuracy	F1 Score	Accuracy	F1 Score
MAE	✓		0.7312 (0.0070)	0.7314 (0.0069)	0.6242 (0.0041)	0.5990 (0.0098)
DINOv2	✓		0.8786 (0.0013)	0.8789 (0.0014)	0.8465 (0.0008)	0.8337 (0.0010)
MSN	✓		0.8275 (0.0016)	0.8279 (0.0017)	0.6113 (0.0030)	0.5616 (0.0032)
ViT-16	✓		0.8519 (0.0021)	0.8521 (0.0021)	0.7304 (0.0007)	0.7054 (0.0015)
ResNet-18	✓		0.8720 (0.0150)	0.8722 (0.0151)	0.9501 (0.0123)	0.9446 (0.0117)
ResNet-50	✓		0.8877 (0.0059)	0.8884 (0.0059)	0.9466 (0.0023)	0.9399 (0.0040)
Zoobot (MaxViT)		✓	0.8790 (0.0022)	0.8796 (0.0023)	0.8922 (0.0065)	0.8847 (0.0066)
Zoobot (ConvNeXT)		✓	0.9360 (0.0009)	0.9365 (0.0009)	0.9600 (0.0061)	0.9550 (0.0062)
Ours (ViT-16)	✓		0.9272 (0.0005)	0.9276 (0.0004)	0.9732 (0.0004)	0.9702 (0.0005)
Ours (ConvNext)	✓		0.9372 (0.0015)	0.9377 (0.0015)	0.9710 (0.0014)	0.9664 (0.0008)