notesum.ai
Published at December 5ALMA: Alignment with Minimal Annotation
cs.CL
cs.LG
Released Date: December 5, 2024
Authors: Michihiro Yasunaga1, Leonid Shamis1, Chunting Zhou1, Andrew Cohen1, Jason Weston1, Luke Zettlemoyer1, Marjan Ghazvininejad1
Aff.: 1Meta FAIR

| Model | Metric: GPT-4o score | Metric: Armo score | ||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|||||||||||||||||
| Llama3 8B Instruct | 8.25 | 8.78 | 8.06 | 8.37 | 7.48 | 8.59 | 8.41 | 8.74 | 8.07 | 8.59 | 8.06 | 8.63 | ||||||||||||||||
| ALMA: Our aligned models | ||||||||||||||||||||||||||||
|
11th round
(= final) |
8.22 | 8.77 | 7.86 | 8.33 | 7.67 | 8.48 | 8.34 | 8.65 | 7.89 | 8.42 | 8.26 | 8.46 | ||||||||||||||||
|
7.94 | 8.55 | 7.60 | 8.18 | 7.18 | 8.20 | 7.78 | 8.27 | 7.36 | 7.89 | 7.54 | 7.87 | ||||||||||||||||
| Llama3 8B Base (3-shot) | 4.00 | 4.44 | 3.58 | 4.29 | 3.37 | 4.36 | 3.95 | 4.33 | 3.67 | 4.30 | 3.71 | 3.74 | ||||||||||||||||