notesum.ai

Published at November 3

cs.LG

cs.AI

cs.CL

Released Date: November 3, 2024

Authors: Zichen Liu¹, Changyu Chen², Chao Du³, Wee Sun Lee⁴, Min Lin³

Aff.: ¹Sea AI Lab, National University of Singapore; ²Sea AI Lab, Singapore Management University; ³Sea AI Lab; ⁴National University of Singapore

Variant	Inference (Test)	Exploration	Learn	Remark
1	$\pi_{\theta}$	passive	$\pi_{\theta}$	Online DAP (Guo et al., 2024)
2	$\pi_{\theta}$	active	$(\pi_{\theta},{\mathcal{R}})$	SEA without ERM sync (Section 4.2.3)
3	$\pi_{\theta}$	active	$(\pi_{\theta}\leftrightarrow{\mathcal{R}})$	SEA
4	$\operatorname{BoN}(\pi_{\theta},{\mathcal{R}})$	passive	$(\pi_{\theta},{\mathcal{R}})$	-
5	$\operatorname{BoN}(\pi_{\theta},{\mathcal{R}})$	active	$(\pi_{\theta},{\mathcal{R}})$	-
6	$\operatorname{BoN}(\pi_{\theta},{\mathcal{R}})$	active	$(\pi_{\theta}\leftrightarrow{\mathcal{R}})$	SEA with Best-of-N sampling
7	$\operatorname{BoN}(\pi_{\mathrm{ref}},{\mathcal{R}})$	active	${\mathcal{R}}$	Not learn policy (Dwaracherla et al., 2024)