notesum.ai

Published at November 25

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?

cs.CL

Released Date: November 25, 2024

Authors: Sohee Yang¹, Nora Kassner¹, Elena Gribovskaya¹, Sebastian Riedel¹, Mor Geva²

Aff.: ¹Google DeepMind, UCL; ²Google Research, Tel Aviv University

Arxiv: http://arxiv.org/abs/2411.16679v1

Refer to caption

$e_{2}$ type	relation composition type	count	example multi-hop query
city	person-birthcity-eventyear	33	$e_{1}$ ’s birth city hosted the Eurovision Song Contest in the year
\@thisrulewidth=\@setrulekerning \@gtempa\futurenonspacelet country	university-locationcountry-anthem	101	The country where $e_{1}$ is in has the national anthem named
	university-locationcountry-isocode	30	The ISO 3166-1 numeric code of the country where $e_{1}$ is located is
	university-locationcountry-year	7	The founding year of the location country of $e_{1}$ is
	person-birthcountry-anthem	22	The name of the national anthem of $e_{1}$ ’s country of birth is
	person-birthcountry-isocode	6	The ISO 3166-1 numeric code used for the country where $e_{1}$ was born is
\@thisrulewidth=\@setrulekerning \@gtempa\futurenonspacelet university	person-undergraduniversity-founder	33	The person who founded $e_{1}$ ’s undergrad university is named
	person-undergraduniversity-year	25	The year when the university where $e_{1}$ studied as an undergrad was founded is
\@thisrulewidth=\@setrulekerning \@gtempa\futurenonspacelet year	person-birthyear-winner	4,484	The winner of the Nobel Prize in Literature in $e_{1}$ ’s birth year was
	city-eventyear-winner	2	In the year when the G7 Summit were hosted in $e_{1}$ , the Nobel Prize in Chemistry was awarded to
	university-inceptionyear-winner	632	In the founding year of $e_{1}$ , the Nobel Prize in Literature was awarded to
	university-inceptionyear-hostleader	9	The person who was the host leader of the G7 Summit in the founding year of $e_{1}$ is
	university-inceptionyear-eventcountry	13	In the year $e_{1}$ was founded, the host country of the G7 Summit was
	university-inceptionyear-eventcity	62	In the founding year of $e_{1}$ , the host city of the G7 Summit was
	person-birthyear-eventcity	1,389	In the birth year of $e_{1}$ , the Winter Olympics were hosted in the city of
	person-birthyear-hostleader	260	The leader of the host of the G7 Summit in $e_{1}$ ’s birth year is
	person-birthyear-eventcountry	124	The country where the Eurovision Song Contest took place in the birth year of $e_{1}$ is
		7,232