notesum.ai
Published at December 4Testing Neural Network Verifiers: A Soundness Benchmark with Hidden Counterexamples
cs.LG
cs.AI
cs.SE
Released Date: December 4, 2024
Authors: Xingjian Zhou1, Hongji Xu2, Andy Xu1, Zhouxing Shi1, Cho-Jui Hsieh1, Huan Zhang3
Aff.: 1University of California, Los Angeles; 2Duke University; 3University of Illinois Urbana-Champaign

| Term | Definition |
|---|---|
| Instance | An instance contains an NN and a property to verify as Eq. (5). |
| Counterexample (CEX) | A counterexample violates the property to verify as Eq. (6). |
| Hidden counterexample | A hidden counterexample is a pre-defined counterexample which satisfies Eq. (6) and cannot be easily found by adversarial attacks as Eq. (7). |
| Unverifiable instance | An unverifiable instance contains a hidden counterexample deliberately planted. |
| Clean instance | A clean instance does not contain a pre-defined counterexample. |
| Verified | An instance is claimed as verified if an NN verifier determines that Eq. (5) provably holds. |
| Falsified | An instance is claimed as falsified if a counterexample which violates Eq. (5) is found. |