notesum.ai
Published at December 3GerPS-Compare: Comparing NER methods for legal norm analysis
cs.CL
cs.AI
Released Date: December 3, 2024
Authors: Sarah T. Bachinger1, Christoph Unger, Robin Erd, Leila Feddoul, Clara Lachenmaier, Sina Zarrieß, Birgitta König-Ries
Aff.: 1Heinz Nixdorf Chair for Distributed Information Systems, Friedrich Schiller University Jena, Germany

| Class | BiLSTM-CRF | XLM-R | LeoLM (opt) | LeoLM (pes) | Rule-based |
|---|---|---|---|---|---|
| Action | 0.7443 | 0.7621 | 0.6102 | 0.0421 | 0.6049 |
| Condition | 0.8244 | 0.8329 | 0.4678 | 0.2240 | 0.5944 |
| Data field | 0.0721 | 0.1676 | 0.1076 | 0.0338 | 0.1829 |
| Document | 0.7661 | 0.8126 | 0.6144 | 0.0178 | 0.5861 |
| Recipient of service | 0.7674 | 0.8004 | 0.6828 | 0.0220 | 0.5531 |
| Deadline | 0.5967 | 0.6569 | 0.4699 | 0.1485 | 0.4813 |
| Legal grounds for action | 0.7985 | 0.8362 | 0.6643 | 0.4794 | 0.4450 |
| Main actor | 0.7315 | 0.7724 | 0.4239 | 0.0129 | 0.5747 |
| Contributor | 0.5276 | 0.6173 | 0.5020 | 0.1227 | 0.4258 |
| Signaling word | 0.8352 | 0.8423 | 0.3940 | 0.0701 | 0.6341 |
| Macro F1-score | 0.6058 | 0.6455 | 0.4488 | 0.1067 | 0.5082 |