notesum.ai
Published at November 29Autocorrelation Matters: Understanding the Role of Initialization Schemes for State Space Models
cs.LG
Released Date: November 29, 2024
Authors: Fusheng Liu1, Qianxiao Li1
Aff.: 1National University of Singapore

| ListOps | Text | Retrieval | Image | Pathfinder | PathX | Avg | |
| Baseline | 60.47 | 86.18 | 89.46 | 88.19 | 93.06 | 91.95 | 84.89 |
| Initialize 10% of to be 0 | 61.44 | 88.05 | 90.73 | 89.11 | 95.58 | 97.55 | 87.08 |
| Ratio for after training | 1.29% | 1.99% | 2.58% | 4.31% | 4.31% | 4.04% | 3.09% |