notesum.ai
Published at November 15Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning
cs.LG
cs.AI
cs.CL
Released Date: November 15, 2024
Authors: Jeffrey Olmo1, Jared Wilson1, Max Forsey1, Bryce Hepner1, Thomas Vin Howe1, David Wingate1
Aff.: 1Brigham Young University

| Sparsity | g-SAE | TopK | E2E |
|---|---|---|---|
| 3 | 0.95 | 0.98 | 0.95 |
| 5 | 0.88 | 0.92 | 0.87 |
| 10 | 0.71 | 0.81 | 0.68 |
| 32 | 0.34 | 0.43 | 0.29 |
| 64 | 0.19 | 0.26 | 0.16 |