notesum.ai

Published at November 7

Scaling Laws for Pre-training Agents and World Models

cs.LG

cs.AI

Released Date: November 7, 2024

Authors: Tim Pearce¹, Tabish Rashid¹, Dave Bignell¹, Raluca Georgescu¹, Sam Devlin¹, Katja Hofmann¹

Aff.: ¹Microsoft Research

Arxiv: http://arxiv.org/abs/2411.04434v1

Refer to caption

	Frontier fit		Parametric fit
Experiment	$N_{\text{optimal}}\propto C^{a}$	$D_{\text{optimal}}\propto C^{b}$	$N_{\text{optimal}}\propto C^{a}$	$D_{\text{optimal}}\propto C^{b}$
WM-Token-256	0.49	0.51	0.52	0.48
WM-Token-540	0.62	0.37	0.78	0.22
BC-Token-540	N/A	N/A	0.32	0.68
BC-CNN	0.66	0.34	0.47	0.53