notesum.ai

Published at December 5

A Context-aware Framework for Translation-mediated Conversations

cs.CL

Released Date: December 5, 2024

Authors: José Pombal¹, Sweta Agrawal², Patrick Fernandes³, Emmanouil Zaranis³, André F. T. Martins⁴

Aff.: ¹Unbabel; ²Instituto de Telecomunicações Instituto Superior Técnico, Universidade de Lisboa; ³Instituto de Telecomunicações; ⁴University of Carnegie Mellon

Arxiv: http://arxiv.org/pdf/2412.04205v1

		en-xx			xx-en
Model	Context?	chrF $\uparrow$	Comet $\uparrow$	MetricX $\downarrow$	chrF $\uparrow$	Comet $\uparrow$	MetricX $\downarrow$
Baselines
GPT-4o	✗	70.09	92.62	0.37	77.33	92.41	0.50
	✓	70.34	92.93	0.33	74.75	91.59	0.58
\cdashline1-9[.4pt/2pt] TowerInstruct	✗	64.95	91.69	0.38	76.04	92.17	0.56
	✓	63.39	91.09	0.49	74.32	91.36	0.60
+ QAD (Comet)	✗	65.20	92.87	0.31	75.59	92.80	0.52
+ QAD (ContextComet)	✗	65.06	92.57	0.31	75.91	92.65	0.51
\cdashline1-9[.4pt/2pt] TowerChat	✗	71.68	93.01	0.32	77.97	92.72	0.51
	✓	75.93	93.63	0.32	78.87	93.01	0.47
+ QAD (Comet)	✓	76.36	94.18	0.25	78.92	93.39	0.44
+ QAD (ContextComet)	✓	76.56	94.05	0.26	78.92	93.24	0.44