notesum.ai

Published at November 8

Training objective drives the consistency of representational similarity across datasets

cs.CV

cs.AI

cs.LG

Released Date: November 8, 2024

Authors: Laure Ciernik¹, Lorenz Linhardt¹, Marco Morik¹, Jonas Dippel², Simon Kornblith³, Lukas Muttenthaler¹

Aff.: ¹Machine Learning Group, Technische Universität Berlin, BIFOLD, Berlin, Germany; ²Aignostics, Machine Learning Group, Technische Universität Berlin, BIFOLD, Berlin, Germany; ³Anthropic, San Francisco, CA, USA

Arxiv: http://arxiv.org/abs/2411.05561v1

Category	Nr. models	Category	Nr. models
Training objective		Training data
Image-Text (Img-Txt)	14	ImageNet-1k (IN1k)	37
Self-Supervised (SSL)	20	ImageNet-21k (IN21k)	9
Supervised (Sup)	30	Large	11
		XLarge	7
Architecture Class		Model Size
Convolutional (CNN)	24	small $<100M$ parameter	32
Transformer (TX)	40	medium $<200M$ parameter	14
		large $<400M$ parameter	8
		xlarge $>400M$ parameter	10