notesum.ai
Published at November 18Topology-aware Preemptive Scheduling for Co-located LLM Workloads
cs.DC
cs.AI
Released Date: November 18, 2024
Authors: Ping Zhang1, Lei Su1, Jinjie Yang1, Xin Chen1
Aff.: 1Baichuan-Inc

| Method | No. Preemptions | No. Hit | Hit Rate |
|---|---|---|---|
| Gödel Standard Preemption | 100 50 | 2225 | 44.5% |
| Gödel + FlexTopo | 100 50 | 5000 | 100% |