|
|
Experiment Customization |
| Type |
Research Question |
# |
Text added to baseline prompt? |
Image provided with prompt? |
| Baseline |
How aligned are LMs with human and COMPAS decisions? How accurate are LMs compared to humans and COMPAS?
|
1 |
None |
None |
| Steerability |
Can a LM be steered towards human or COMPAS decisions? Does steering improve LLM accuracy?
|
1 |
None |
None |
| 2 |
{With_Humans_Decisions} |
| 3 |
{With_Humans_Decisions} |
| 4 |
{With_COMPAS_Score} |
|
5 |
{With_Humans_Decisions} + {With_COMPAS_Score}
|
| Multimodal |
How does adding a photo affect alignment and accuracy?
|
1 |
None |
None |
| 2 |
{Here-Is-A-Photo} |
Hypothetical defendant photo |
| 3 |
{Here-Is-A-Photo} |
Placeholder photo |
| Bias |
How do bias mitigation techniques such as anti-discrimination prompting and moderation affect alignment and accuracy? Is bias reduced?
|
1 |
None |
None |
| 2 |
{Here-Is-A-Photo} |
Hypothetical defendant photo |
| 3 |
{Illegal-Ignore} |
None |
|
|
4 |
{Here-Is-A-Photo} + {Illegal-Ignore}
|
Hypothetical defendant photo |
