notesum.ai
Published at November 29MIMDE: Exploring the Use of Synthetic vs Human Data for Evaluating Multi-Insight Multi-Document Extraction Tasks
cs.CL
Released Date: November 29, 2024
Authors: John Francis1, Saba Esnaashari1, Anton Poletaev1, Sukankana Chakraborty2, Youmna Hashem1, Jonathan Bright1
Aff.: 1Public Policy Programme, The Alan Turing Institute, British Library, UK; 2Public Policy Programme, The Alan Turing Institute, British Library, UK; Centre for Advanced Spatial Analysis, University College London, UK
| Question | Human Data | Synthetic Data | ||||
| #Insights | Insights/ Response | Words/ Response | #Insights | Insights/ Response | Words/ Response | |
| To what extent do you agree or disagree with the changes to the highway code which states that: drivers and motorcyclists should switch off their mobile phone before setting off? (Strongly agree / agree / disagree / strongly disagree / Neither agree nor disagree) Please explain your answer. | 12 | 1.3 | 167.3 | 11 | 2.0 | 239.4 |
| The UK government is planning to ban parking on the pavement everywhere in the UK. Do you think this change would result in more positive or negative impacts with regards to accessibility? Please explain. | 13 | 1.3 | 168 | 14 | 2.0 | 223.7 |
| Please rank what you think the speed limit for inner-city (built-up) areas should be: 20MPH / 30MPH / 40MPH Please explain your answer. | 15 | 1.3 | 139.5 | 14 | 2.0 | 221.1 |
| How would you describe your experience with mobile internet (e.g. internet on your cell phone) in the UK? | 8 | 1.3 | 118.4 | 6 | 2.0 | 219.6 |
| For UK Parliamentary elections, the current voting age is 18. I think the voting age should be lowered to … 16 / stay at 18 / be raised to 21. Please explain. | 12 | 1.2 | 146.6 | 15 | 2.0 | 229.0 |