notesum.ai
Published at November 29COLD: Causal reasOning in cLosed Daily activities
cs.CL
cs.AI
cs.LG
Released Date: November 29, 2024
Authors: Abhinav Joshi1, Areeb Ahmad1, Ashutosh Modi1
Aff.: 1Department of Computer Science and Engineering, Indian Institute of Technology Kanpur (IIT Kanpur), Kanpur, India

| Triplets | Model Name | cake | shopping | train | tree | bus |
|---|---|---|---|---|---|---|
| causal triplets | gpt-neo-125M | 50.71 | 50.01 | 49.99 | 50.13 | 50.15 |
| gpt-neo-1.3B | 44.77 | 45.69 | 42.52 | 45.67 | 42.89 | |
| gemma-2b | 53.76 | 52.19 | 60.57 | 60.71 | 53.64 | |
| gpt-neo-2.7B | 50.00 | 50.01 | 50.00 | 50.01 | 50.00 | |
| phi-2 | 85.14 | 83.65 | 77.29 | 82.24 | 71.74 | |
| gpt-j-6B | 49.59 | 50.02 | 50.29 | 49.92 | 49.93 | |
| Llama-2-7b-chat-hf | 77.92 | 72.41 | 73.48 | 72.40 | 68.21 | |
| Mistral-7B-v0.1 | 77.64 | 69.38 | 68.46 | 72.43 | 69.37 | |
| gemma-7b | 81.47 | 82.26 | 77.24 | 80.78 | 70.29 | |
| Meta-Llama-3-8B | 80.79 | 76.46 | 76.08 | 78.21 | 67.39 |