notesum.ai

Published at November 12

Towards Low-bit Communication for Tensor Parallel LLM Inference

cs.AI

cs.LG

Released Date: November 12, 2024

Authors: Harry Dong¹, Tyler Johnson², Minsik Cho², Emad Soroush²

Aff.: ¹Carnegie Mellon University; ²Apple

Arxiv: http://arxiv.org/abs/2411.07942v1

Refer to caption