notesum.ai

Published at October 21

Subword Embedding from Bytes Gains Privacy without Sacrificing Accuracy and Complexity

cs.LG

cs.AI

Released Date: October 21, 2024

Authors: Mengjiao Zhang¹, Jia Xu¹

Aff.: ¹Stevens Institute of Technology

Arxiv: https://arxiv.org/abs/2410.16410v1

Refer to caption

Byte Tokens per Subword ( $n$ )	Byte Vocabulary Size ( $V_{b}$ )
Byte Tokens per Subword ( $n$ )	64	128	256
4	0.79M	1.05M	1.57M
8	1.05M	1.57M	2.62M
16	1.57M	2.62M	4.72M