notesum.ai
Collect (0)
Published at November 4
Share
Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoders
cs.LG
cs.AI
Released Date: November 4, 2024
Authors:
Kola Ayonrinde
Arxiv:
http://arxiv.org/abs/2411.02124v1
SAE Type
6k latent dim
34m dim
16m dim
Standard SAE
(Bricken et al.,
2023
)
9.0%
-
>90.0%
SAE++ (w/ resampling)
(Templeton et al.,
2024
)
5.1 %
64.7%
-
Token Choice (TopK) SAE
(Gao et al.,
2024
)
0.0%
-
7.0%
Feature Choice SAE (Ours)
0%
0%
0%
