notesum.ai
Published at November 8SM3-Text-to-Query: Synthetic Multi-Model Medical Text-to-Query Benchmark
cs.DB
cs.AI
Released Date: November 8, 2024
Authors: Sithursan Sivasubramaniam1, Cedric Osei-Akoto1, Yi Zhang1, Kurt Stockinger1, Jonathan Fuerst
Aff.: 1Zurich University of Applied Sciences, Switzerland

| EHRSQL | MIMICSQL | SM3Cypher | SM3MQL | SM3SPARQL | SM3SQL | |
|---|---|---|---|---|---|---|
| Tokens | 32.63 | 19.45 | 7.28 | 28.56 | 31.35 | 9.93 |
| Keywords | 11.74 | 6.38 | 2.43 | 5.08 | 7.41 | 4.06 |
| Joins/Traversals | 0.33 | 0.64 | 0.62 | 0.04 | 4.00 | 0.31 |
| Nesting Depth | 1.37 | 0.77 | 2.14 | 3.77 | 1.15 | 0.15 |