notesum.ai

Published at November 12

BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks

cs.MA

cs.AI

cs.CL

cs.LG

68T42

I.2.1; I.2.2; I.2.5; I.2.7; I.2.8

Released Date: November 12, 2024

Authors: Shubham Gandhi¹, Manasi Patwardhan, Lovekesh Vig², Gautam Shroff

Aff.: ¹TCS Research Pune, India; ²TCS Research Delhi, India

Arxiv: http://arxiv.org/abs/2411.07464v1

Single Agent(Huang et al., 2023)

Profiling + Cascade

Profiling + Cascade + Expert

Task

G + R

G

C + R

C

Ge + R

Ge

Ge + Ch + R

Ge + Ch

Ge + G + R

Ge + G

cifar10

25

($0.583)

50

($0.44)

8

($0.058)

48

($0.044)

12.5

($0)*

($0.016)

0

($0)*

($0.044)

62.5

($0)*

($0.025)

25

($0)*

($0.093)

75

($0.057)*

($0.094)

37.5

($0.06)*

($0.034)

imdb

12.5

($1.48)

25

($0.919)

0

($0.147)

0

($0.091)

0

($0)*

($0.006)

0

($0)*

($0.038)

0

($0)*

($0.022)

0

($0)*

($0.023)

0

($0.014)*

($0.060)

0

($0.071)*

($0.124)

ogbn-arxiv

50

($1.27)

87.5

($1.112)

40

($0.125)

32

($0.109)

25

($0)*

($0.011)

25

($0)*

($0.016)

50

($0)*

($0.020)

50

($0)*

($0.015)

50

($0.033)*

($0.067)

75

($0.026)*

($0.045)

house-price

25

($1.6)

12.5

($0.938)

64

($0.158)

76

($0.093)

25

($0)*

($0.042)

37.5

($0)*

($0.070)

62.5

($0)*

($0.046)

62.5

($0)*

($0.049)

87.5

($0.091)*

($0.161)

75

($0.038)*

($0.153)

spaceship-

titanic

25

($1.42)

12.5

($0.85)

4

($0.141)

16

($0.088)

37.5

($0)*

($0.039)

37.5

($0)*

($0.052)

75

($0)*

($0.077)

100

($0.0004)*

($0.038)

75

($0.021)*

($0.096)

100

($0.091)*

($0.213)

parkinsons-

disease

12.5

($2.9)

0

($1.57)

0

($0.287)

0

($0.155)

0

($0)*

($0.078)

0

($0)*

($0.065)

0

($0)*

($0.024)

0

($0)*

($0.028)

0

($0.099)*

($0.186)

0

($0.107)*

($0.183)

feedback

37.5

($1.25)

12.5

($1.15)

0

($0.124)

0

($0.114)

0

($0)*

($0.058)

0

($0)*

($0.046)

0

($0.0005)*

($0.038)

0

($0)*

($0.030)

0

($0.047)*

($0.110)

0

($0.022)*

($0.068)

llama-

inference

0

($1.46)

0

($0.927)

0

($0.145)

0

($0.092)

0

($0)*

($0.030)

0

($0)*

($0.059)

0

($0.0003)*

($0.032)

0

($0)*

($0.034)

0

($0.027)*

($0.074)

0

($0.055)*

($0.093)

vectorization

0

($1.16)

0

($1.23)

0

($0.114)

0

($0.121)

0

($0)*

($0.062)

12.5

($0)*

($0.058)

0

($0)*

($0.039)

12.5

($0)*

($0.025)

0

($0.017)*

($0.077)

75

($0.004)*

($0.092)

CLRS

12.5

($0.523)

50

($0.43)

52

($0.052)

40

($0.042)

0

($0)*

($0.006)

0

($0)*

($0.006*)

0

($0.0003)*

($0.033)

12.5

($0)*

($0.022)

0

($0.027)*

($0.050)

0

($0.046)*

($0.138)

babylm

0

($0.818)

0

($0.637)

0

($0.081)

0

($0.063)

0

($0)*

($0.032)

0

($0)*

($0*.027)

0

($0.0025)*

($0.046)

0

($0.001)*

($0.024)

0

($0.086)*

($0.147)

0

($0.078)*

($0.113)

Average

18.18

($1.315)

22.72

($0.931)

15.27

($0.13)

20

($0.092)

9.09

($0)*

($0.034)

10.23

($0)*

($0.044)

22.72

($0.0003)*

($0.036)

22.72

($0.0001)*

($0.032)

26.14

($0.047)*

($0.102)

32.95

($0.054)*

($0.120)