notesum.ai

Published at November 4

INQUIRE: A Natural World Text-to-Image Retrieval Benchmark

cs.CV

cs.AI

cs.CL

cs.IR

Released Date: November 4, 2024

Authors: Edward Vendrow¹, Omiros Pantazis², Alexander Shepard³, Gabriel Brostow², Kate E. Jones², Oisin Mac Aodha⁴, Sara Beery¹, Grant Van Horn⁵

Aff.: ¹Massachusetts Institute of Technology; ²University College London; ³iNaturalist; ⁴University of Edinburgh; ⁵University of Massachusetts Amherst

Arxiv: http://arxiv.org/abs/2411.02537v1

Embedding models
Method	AP	nDCG	MRR
Random	22.1	52.6	0.35
CLIP ViT-B-32 Radford et al. [2021]	30.2	59.1	0.47
CLIP ViT-L-14 Radford et al. [2021]	36.8	64.2	0.57
CLIP ViT-H-14 Fang et al. [2024]	42.6	68.7	0.66
SigLIP SO400m-14 Zhai et al. [2023]	50.1	73.5	0.72
Proprietary multimodal models
GPT-4V Achiam et al. [2023]	47.8	71.9	0.70
GPT-4o OpenAI [2024]	59.6	78.9	0.78