notesum.ai

Published at November 4

Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models

cs.CL

cs.AI

cs.CE

cs.LG

Released Date: November 4, 2024

Authors: Jonas Zausinger¹, Lars Pennig¹, Kacper Chlodny¹, Vincent Limbach¹, Anna Ketteler¹, Thorben Prein¹, Vishwa Mohan Singh², Michael Morris Danziger³, Jannis Born³

Aff.: ¹TU Munich, Germany; TUM.AI, Germany; ²TUM.AI, Germany; LMU Munich, Germany; ³IBM Research Europe, Switzerland

Arxiv: http://arxiv.org/abs/2411.02083v1

Model	Acc.	MAE	R²
Standard T5	.6448	.1303	.9688
Standard + NTL-MSE	.7189	.1091	.9739
Standard + NTL-WAS	.7460	.0980	.9766
RT	.7136	.1135	.9701
RT + NTL-MSE	.6990	.1291	.9580
xVal	.0000	.2581	.9735