Automatic Quality Management in Crowdsourcing

Crowdsourcing platform management must provide its human workers to crowdsourcing customers based on their current needs and requirements. Task-based crowdsourcing platforms such as Amazon Mechanical Turk (AMT) enable businesses to access the human-power of thousands of people on demand by posting human-task requests on Amazon's web site. AMT has a set of built-in capabilities addressing fundamental requirements of online businesses. Automated management allows handling the growing number of crowdsourcing platforms. CaaS (Crowds as a Service) is able to perform many management tasks automatically, which have to be done manually in other systems. CaaS employs a market-based approach that uses auctions in order to implement these design principles. Based on the bids price and the suitability of the worker, a ranking is created. CaaS then assigns the task to the top-ranked worker. Policies are used for goal-driven management of CaaS specifying the expected quality of task-processing and profit considerations.