notesum.ai
Published at November 19Balancing Accuracy and Efficiency in Multi-Turn Intent Classification for LLM-Powered Dialog Systems in Production
cs.CL
cs.AI
cs.IR
Released Date: November 19, 2024
Authors: Junhua Liu1, Yong Keat Tan2, Bin Fu2, Kwan Hui Lim3
Aff.: 1Forth AI; 2Shopee; 3Singapore University of Technology and Design

| MKT | Model | CL-Label | Accuracy | |
|---|---|---|---|---|
| SG | Naive Concat. | - | - | 60.52% |
| SG | Selective Concat. | - | - | 56.99% |
| SG | Llama2-7B | ✘ | ✘ | 56.24% |
| SG | Llama2-7B | ✔ | ✘ | 61.33% |
| SG | Domain-Llama2-7B | ✔ | ✘ | 63.23% |
| ID | Naive Concat. | - | - | 60.61% |
| ID | Selective Concat. | - | - | 63.23% |
| ID | Llama2-7B | ✔ | ✘ | 49.96% |
| ID | SeaLLM-7B-chat | ✔ | ✘ | 52.49% |
| ID | SeaLLM-7B-chat | ✔ | ✔ | 55.02% |