notesum.ai
Published at December 5ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation
cs.CV
cs.LG
Released Date: December 5, 2024
Authors: Dayoung Gong1, Suha Kwak1, Minsu Cho1
Aff.: 1POSTECH
| methods | 50 Salads [58] | Breakfast [36] | GTEA [21] | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1@{10, 25, 50} | edit | Acc. | Avg. | F1@{10, 25, 50} | edit | Acc. | Avg. | F1@{10, 25, 50} | Edit | Acc. | Avg. | |
| MS-TCN [19] | 76.3 / 74.0 / 64.5 | 67.9 | 80.7 | 72.7 | 52.6 / 48.1 / 37.9 | 61.7 | 66.3 | 53.3 | 85.8 / 83.4 / 69.8 | 79.0 | 76.3 | 78.9 |
| MS-TCN++ [41] | 80.7 / 78.5 / 70.1 | 74.3 | 83.7 | 77.5 | 64.1 / 58.6 / 45.9 | 65.6 | 67.6 | 60.4 | 88.8 / 85.7 / 76.0 | 83.5 | 80.1 | 82.8 |
| SSTDA [12] | 83.0 / 81.5 / 73.8 | 75.8 | 83.2 | 79.5 | 75.0 / 69.1 / 55.2 | 73.7 | 70.2 | 68.6 | 90.0 / 89.1 / 78.0 | 86.2 | 79.8 | 84.6 |
| GTRM [30] | 75.4 / 72.8 / 63.9 | 67.5 | 82.6 | 72.4 | 57.5 / 54.0 / 43.3 | 58.7 | 65.0 | 55.7 | - / - / - | - | - | - |
| BCN [61] | 82.3 / 81.3 / 74.0 | 74.3 | 84.4 | 79.3 | 68.7 / 65.5 / 55.0 | 66.2 | 70.4 | 65.2 | 88.5 / 87.1 / 77.3 | 84.4 | 79.8 | 83.4 |
| MTDA [11] | 82.0 / 80.1 / 72.5 | 75.2 | 83.2 | 78.6 | 74.2 / 68.6 / 56.5 | 73.6 | 71.0 | 68.8 | 90.5 / 88.4 / 76.2 | 85.8 | 80.0 | 84.2 |
| Global2local [23] | 80.3 / 78.0 / 69.8 | 73.4 | 82.2 | 76.7 | 74.9 / 69.0 / 55.2 | 73.3 | 70.7 | 68.6 | 89.9 / 87.3 / 75.8 | 84.6 | 78.5 | 83.2 |
| HASR [3] | 86.6 / 85.7 / 78.5 | 81.0 | 83.9 | 83.1 | 74.7 / 69.5 / 57.0 | 71.9 | 69.4 | 68.5 | 90.9 / 88.6 / 76.4 | 87.5 | 78.7 | 84.4 |
| ASRF [32] | 84.9 / 83.5 / 77.3 | 79.3 | 84.5 | 81.9 | 74.3 / 68.9 / 56.1 | 72.4 | 67.6 | 67.9 | 89.4 / 87.8 / 79.8 | 83.7 | 77.3 | 83.6 |
| ASFormer [65] | 85.1 / 83.4 / 76.0 | 79.6 | 85.6 | 81.9 | 76.0 / 70.6 / 57.4 | 75.0 | 73.5 | 70.5 | 90.1 / 88.8 / 79.2 | 84.6 | 79.7 | 84.5 |
| ASFormer + KARI [24] | 85.4 / 83.8 / 77.4 | 79.9 | 85.3 | 82.4 | 78.8 / 73.7 / 60.8 | 77.8 | 74.0 | 73.0 | - / -/ - | - | - | - |
| Temporal Agg. [52] | - / - / - | - | - | - | 59.2 / 53.9 / 39.5 | 54.5 | 64.5 | 54.3 | - / - / - | - | - | - |
| UARL [10] | 85.3 / 83.5 / 77.8 | 78.2 | 84.1 | 81.8 | 65.2 / 59.4 / 47.4 | 66.2 | 67.8 | 61.2 | 92.7 / 91.5 / 82.8 | 88.1 | 79.6 | 86.9 |
| DPRN [46] | 87.8 / 86.3 / 79.4 | 82.0 | 87.2 | 84.5 | 75.6 / 70.5 / 57.6 | 75.1 | 71.7 | 70.1 | 92.9 / 92.0 / 82.9 | 90.9 | 82.0 | 88.1 |
| SEDT [35] | 89.9 / 88.7 / 81.1 | 84.7 | 86.5 | 86.2 | - / - / - | - | - | - | 93.7 / 92.4 / 84.0 | 91.3 | 81.3 | 88.5 |
| TCTr [4] | 87.5 / 86.1 / 80.2 | 83.4 | 86.6 | 84.8 | 76.6 / 71.1 / 58.5 | 76.1 | 77.5 | 72.0 | 91.3 / 90.1 / 80.0 | 87.9 | 81.1 | 86.1 |
| FAMMSDTN [17] | 86.2 / 84.4 / 77.9 | 79.9 | 86.4 | 83.0 | 78.5 / 72.9 / 60.2 | 77.5 | 74.8 | 72.8 | 91.6 / 90.9 / 80.9 | 88.3 | 80.7 | 86.5 |
| DTL [63] | 87.1 / 85.7 / 78.5 | 80.5 | 86.9 | 83.7 | 78.8 / 74.5 / 62.9 | 77.7 | 75.8 | 73.9 | - / - / - | - | - | - |
| UVAST [8] | 89.1 / 87.6 / 81.7 | 83.9 | 87.4 | 85.9 | 76.9 / 71.5 / 58.0 | 77.1 | 69.7 | 70.6 | 92.7 / 91.3 / 81.0 | 92.1 | 80.2 | 87.5 |
| BrPrompt [40] | 89.2 / 87.8 / 81.3 | 83.8 | 88.1 | 86.0 | - / - / - | - | - | - | 94.1 / 92.0 / 83.0 | 91.6 | 81.2 | 88.4 |
| MCFM [31] | 90.6 / 89.5 / 84.2 | 84.6 | 90.3 | 87.8 | - / - / - | - | - | - | 91.8 / 91.2 / 80.8 | 88.0 | 80.5 | 86.5 |
| LTContext [5] | 89.4 / 87.7 / 82.0 | 83.2 | 87.7 | 86.0 | 77.6 / 72.6 / 60.1 | 77.0 | 74.2 | 72.3 | - / - / - | - | - | - |
| DiffAct [43] | 90.1 / 89.2 / 83.7 | 85.0 | 88.9 | 87.4 | 80.3 / 75.9 / 64.6 | 78.4 | 76.4 | 75.1 | 92.5 / 91.5 / 84.7 | 89.6 | 82.2 | 88.1 |
| ActFusion (ours) | 91.6 / 90.7 / 84.8 | 86.0 | 89.3 | 88.5 | 81.0 /76.2 / 64.7 | 79.3 | 76.4 | 75.5 | 94.1 / 93.3 / 86.9 | 91.6 | 81.9 | 89.6 |