notesum.ai

Published at November 29

Scaling Transformers for Low-Bitrate High-Quality Speech Coding

eess.AS

cs.AI

cs.LG

cs.SD

eess.SP

Released Date: November 29, 2024

Authors: Julian D Parker, Anton Smirnov¹, Jordi Pons¹, CJ Carr¹, Zack Zukowski¹, Zach Evans¹, Xubo Liu¹

Aff.: ¹Stability AI

Arxiv: http://arxiv.org/pdf/2411.19842v1

Refer to caption

Model	BPS	TPF	TPS	SISDR $\uparrow$	Mel $\downarrow$	STFT $\downarrow$	PESQ $\uparrow$	STOI $\uparrow$	MOSNet $\uparrow$
DAC	$1000$	$2$	$100$	$-6.51$	$1.49$	$1.76$	$1.64$	$0.75$	$2.77$
	$2000$	$4$	$200$	$-0.37$	$1.07$	$1.41$	$2.29$	$0.85$	$2.95$
Encodec	$1500$	$2$	$150$	$-0.22$	$1.14$	$1.49$	$2.36$	$0.85$	$2.87$
	$3000$	$4$	$300$	$2.77$	$0.95$	$1.33$	$2.84$	$0.90$	$2.98$
SpeechTokenizer	$1000$	$2$	$100$	$-3.30$	$1.06$	$1.37$	$2.41$	$0.85$	$2.94$
	$1500$	$3$	$150$	$-1.33$	$0.91$	$1.25$	$2.70$	$0.88$	$3.10$
SemantiCodec	$337.5$	$2$	$25$	–	$1.20$	$1.53$	$2.21$	$0.79$	$3.24$
	$675$		$50$	–	$0.98$	$1.32$	$2.65$	$0.86$	$3.29$
Mimi	$550$	$4$	$50$	$-4.45$	$1.19$	$1.55$	$2.48$	$0.85$	$3.11$
	$1100$	$8$	$100$	$2.20$	$0.94$	$1.31$	$3.01$	$0.90$	$3.24$
TAAE	$400$	$1$	$25$	$3.18$	$0.97$	$1.35$	$2.96$	$0.90$	$3.36$
	$700$	$2$	$50$	$4.73$	$0.86$	$1.26$	$3.09$	$0.92$	$3.36$
+ no quant.	$-$	$-$	$-$	$5.08$	$0.85$	$1.25$	$3.12$	$0.92$	$3.36$