notesum.ai

Published at November 14

Harnessing multiple LLMs for Information Retrieval: A case study on Deep Learning methodologies in Biodiversity publications

cs.IR

cs.AI

H.3.3; I.2.7

Released Date: November 14, 2024

Authors: Vamsi Krishna Kommineni¹, Birgitta König-Ries, Sheeba Samuel²

Aff.: ¹Heinz Nixdorf Chair for Distributed Information Systems, Friedrich Schiller University Jena, Jena, Germany; German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, Leipzig, Germany; Max Planck Institute for Biogeochemistry, Jena, Germany; ²Distributed and Self-organizing Systems, Chemnitz University of Technology, Chemnitz, Germany

Arxiv: http://arxiv.org/abs/2411.09269v1

LLM name	Agreements between LLM and human response	Cohen’s kappa score (IAA)
Mixtral 8x22B Instruct v0.1	667/840	0.5711
Mixtral 8x7B	666/840	0.5583
Llama 3.1 70B	735/840	0.7221
Llama 3 70B	752/840	0.7708
Gemma 2 9B	746/840	0.7128