notesum.ai

Published at November 7

SuffixDecoding: A Model-Free Approach to Speeding Up Large Language Model Inference

cs.CL

cs.AI

cs.DC

cs.LG

Released Date: November 7, 2024

Authors: Gabriele Oliaro¹, Zhihao Jia², Daniel Campos¹, Aurick Qiao¹

Aff.: ¹Snowflake AI Research; ²Carnegie Mellon University

Arxiv: http://arxiv.org/abs/2411.04975v1

Refer to caption