notesum.ai

Published at November 12

Entropy Controllable Direct Preference Optimization

cs.LG

cs.AI

cs.CL

Released Date: November 12, 2024

Authors: Motoki Omura¹, Yasuhiro Fujita², Toshiki Kataoka²

Aff.: ¹The University of Tokyo; ²Preferred Networks, Inc.

Arxiv: http://arxiv.org/abs/2411.07595v1

Refer to caption

	GSM8K $\uparrow$	HumanEval $\uparrow$	MMLU-Pro $\uparrow$	IFEval $\uparrow$
DPO ( $\alpha=1$ )	26.40 _±1.76	28.77 _±0.45	31.83 _±0.17	59.63 _±0.72
\hdashlineH-DPO ( $\alpha=0.95$ )	27.77 _±1.39	30.70 _±0.39	32.37 _±0.03	60.17 _±0.34
H-DPO ( $\alpha=0.9$ )	28.83 _±2.32	29.63 _±0.45	32.30 _±0.17	60.93 _±0.50
H-DPO ( $\alpha=0.8$ )	28.66 _±1.23	27.77 _±0.67	31.93 _±0.19	59.90 _±0.59