notesum.ai
Published at November 26RoboPEPP: Vision-Based Robot Pose and Joint Angle Estimation through Embedding Predictive Pre-Training
cs.RO
cs.CV
Released Date: November 26, 2024
Authors: Raktim Gautam Goswami1, Prashanth Krishnamurthy1, Yann LeCun1, Farshad Khorrami1
Aff.: 1New York University

| Method | Known Joint Angles | Known Bounding Box | Panda | Kuka | Baxter | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Synthetic | Real | Synthetic | Synthetic | ||||||||
| Photo | DR | AK | XK | RS | ORB | Photo | DR | DR | |||
| DREAM-F | Yes | No | 79.5 | 81.3 | 68.9 | 24.4 | 76.1 | 61.9 | - | - | - |
| DREAM-Q | Yes | No | 74.3 | 77.8 | 52.4 | 37.5 | 78.0 | 57.1 | - | - | 75.5 |
| DREAM-H | Yes | No | 81.1 | 82.9 | 60.5 | 64.0 | 78.8 | 69.1 | 72.1 | 73.3 | - |
| HPE | No | Yes | 82.0 | 82.7 | 82.2 | 76.0 | 75.2 | 75.2 | 73.9 | 75.1 | 58.8 |
| RoboPose | No | No | 79.7 | 82.9 | 70.4 | 77.6 | 74.3 | 70.4 | 73.2 | 80.2 | 32.7 |
| HPEā | No | No | 40.7 | 41.4 | 66.7 | - | 49.1 | 51.6 | 56.7 | 56.2 | 9.8 |
| RoboPEPP (Ours) | No | No | 84.1 | 83.0 | 75.3 | 78.5 | 80.5 | 77.5 | 76.1 | 76.2 | 34.4 |