notesum.ai

Published at November 29

DeMo: Decoupled Momentum Optimization

cs.LG

cs.AI

Released Date: November 29, 2024

Authors: Bowen Peng¹, Jeffrey Quesnelle¹, Diederik P. Kingma¹

Aff.: ¹Nous Research

Arxiv: http://arxiv.org/pdf/2411.19870v1