notesum.ai

Published at December 5

Quality Control in Open-Ended Crowdsourcing: A Survey

cs.HC

cs.DC

H.1.2; H.4.0

Released Date: December 5, 2024

Authors: Lei Chai¹, Hailong Sun¹, Jing Zhang¹

Aff.: ¹Beihang University

Arxiv: http://arxiv.org/pdf/2412.03991v1

Name	Data Type	Task Type	Open-ended Task	Answer	Size
f-BRS [117]	image	IIP	semantic segmentation	clicks on the positive and negative area	large-un
PDMRR [9]	image	IIP	human body annotation	keypoints on the human skeleton	large-co
CrowdTruth [116]	image	IIP	image description	event description according to the image	large-un
LATTE [198]	image	IIP	LiDAR	clicks on the center of the target object	large-un
Lean [88]	image	IIP	object selection	2-D bounding box on the target object	large-co
Cascade [39]	image	CSDm	taxonomy creation	suggested categories of the colors	large-co
Gadiraju et al. [19]	image	IIP	image transcription	decipher characters from a given image	large-un
Trainbot [56]	text	IIP/CSDm	question-answering	answers to open-ended questions	large-un
BAU [78]	text	IIP	sequence annotation	the text span of the target content	large-un
MAS [78]	text	IIP	sentence translation	translations of the initial sentence	large-un
Nguyen et al. [217]	text	CSDm	opinion aggregation	textual ideas	large-un
Context trees [61]	text	CSDm	importance rating	importance value of event fragments	countable
Kaur et al. [65]	text	CI	plan creation	textual plans	large-un
CrowDEA [107]	text	CI	ideas ranking	ideas and opinions	large-un
Trainbot [56]	text	CSDm/IIP	conversation	multiple rounds answers	large-un
Nguyen et al. [68]	text	CSDm	information finding	middle-names of personalities	large-un
GONCALVES et al. [100]	text	IIP/CSDm	knowledge eliciting	pairs of options and criteria	large-un
CrowdTruth [116]	text	IIP/CSDm	NER	relation between entities	countable
QAMR [12]	text	IIP/CSDm	semantic mining	question-answer pairs	large-un
Heteroglossia [67]	text	CI	story ideation	suggestion of story ideas	large-un
TAS [16]	text	CSDm	news writing	pieces of events	large-un
FFV [20]	text	IIP/CSDm	sentence correction	revised sentence	large-un
VoxDIY [220]	audio	IIP/CSDm	audio transcription	textual content of speech	large-un
Lipping et al. [221]	audio	IIP/CSDm	audio captioning	textual summary of speech content	large-un
Kaspar et al. [222]	video	IIP	video segmentation	scribbles on the video frame	large-un
CrowdTruth [116]	video	IIP/CSDm	video description	textual descriptions of video content	large-un
C-Reference [7]	video	IIP	object selection	2-D bounding box	large-co
Event anchor [8]	video	IIP/CSDm	context of object and event	2-D bounding box	large-un
Popup [199]	video	IIP	position identification	dimension lines on objects	large-co
PDMRR [9]	numerical	IIP/CSDm	id ranking	numerical sequence	countable