notesum.ai

Published at November 11

Spiking Transformer Hardware Accelerators in 3D Integration

cs.NE

cs.AR

Released Date: November 11, 2024

Authors: Boxun Xu¹, Junyoung Hwang², Pruek Vanna-iampikul³, Sung Kyu Lim², Peng Li¹

Aff.: ¹Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA; ²Department of Electrical and Computer Engineering, Georgia Institute of Technology, GA, USA; ³Department of Electrical and Computer Engineering, Georgia Institute of Technology, GA, USA; Department of Electrical Engineering, Burapha University, Chonburi, Thailand

Arxiv: http://arxiv.org/abs/2411.07397v1

Array size H $\times$ W, weight/synaptic integration bitwidth	$16\times 128$ , 8b/16b		$64\times 16$ , 8b/16b		$64\times 16$ , 4b/12b
	2D	3D	2D	3D	2D	3D
Effective Frequency ( $GHz$ )	1.57	1.68	1.68	1.79	1.76	1.85
Area Footprint ( $mm^{2}$ )	0.45 $\times$ 0.9	0.45 $\times$ 0.45	0.45 $\times$ 0.78	0.45 $\times$ 0.4	0.45 $\times$ 0.78	0.45 $\times$ 0.4
Number of Cells	152,335	152,012	88,838	88,447	83,931	83,923
Wire length(m)	1.37	1.10	1.17	0.99	1.00	0.81
Internal Power (mW)	334	310	221.2	215.8	201.2	186.2
Switching Power (mW)	152	137	118.1	107.0	101.2	86.0
Leakage Power (mW)	30.0	29.1	22.8	21.0	19.1	14.4
Total Power (mW)	516	476.1	362.1	343.8	321.5	286.6
Memory Access Latency (ps)	82	26	77	19	80	58
Memory Access Power (mW)	4.17	1.27	4.6	1.3	4.4	0.99