notesum.ai

Published at December 4

From Words to Workflows: Automating Business Processes

cs.AI

Released Date: December 4, 2024

Authors: Laura Minkova¹, Jessica López Espejel, Taki Eddine Toufik Djaidja, Walid Dahhane, El Hassane Ettifouri

Aff.: ¹Novelis Research and Innovation Lab, France

Arxiv: http://arxiv.org/pdf/2412.03446v1

Refer to caption

Score	Description
1	Perfect JSON output. Coherent and semantically similar to the gold standard. Minor syntactical differences are permitted.
0.75	The JSON output has expected structure, though minor reasoning errors are made. e.g erroneous API parameters, variable names are not prefixed with ${} in strings.
0.5	The JSON output is logical, but has considerable weaknesses in its structure. e.g missing Loop structure, hallucinating a non-existent nextStepId, misuse of data extraction, incorrect function for API type step.
0.25	The JSON output is comprehensible given the user request but there are several mistakes or a single big mistake. e.g combination of mistakes such as those outlined above, hallucinating parameter keys or function names.
0	The JSON output is not representative of the user request, lacks a great deal of information, or is nonsensical.