notesum.ai

Published at November 22

Released Date: November 22, 2024

Authors: Fengyuan Liu¹, Nikhil Kandpal¹, Colin Raffel¹

Aff.: ¹University of Toronto & Vector Institute

Method

Method Parameters

Theoretical FLOPs

Speedup over LOO

LOO

N/A

2PT|C|(|C|-1)

1

KV Caching

N/A

PT|C|(|C|-1)

2

Proxy

P^{\prime}:\text{Proxy model size}

2P^{\prime}T|C|(|C|-1)

\frac{P}{P^{\prime}}

Pruning

P^{\prime}:\text{Proxy model size}

\alpha:\text{Fraction of sources}

2PT|C|((\alpha^{2}+\frac{P^{\prime}}{P})|C|-\alpha-\frac{P^{\prime}}{P})

\frac{P(|C|-1)}{(\alpha^{2}P+P^{\prime})|C|-\alpha P-P^{\prime}}

Hierarchical

H:\text{Sources per group}

\beta:\text{Fraction of groups}

2PT|C|((\beta^{2}+\frac{1}{H})|C|-\beta-1)

\frac{H(|C|-1)}{(\beta^{2}H+1)|C|-\beta H-H}