CrowdAdvisor: A Framework for Freelancer Assessment in Online Marketplace

Hiring is one of the important challenges in the context of online labor marketplace. Unlike traditional hiring, where workers are hired either as a full time employee or as a contractor, hiring from online marketplaces are done for individual jobs of short duration. As these marketplaces are open for anyone, hiring becomes challenging due to the large number of freelancers applying for a posted job. Quite often, clients use ratings of the freelancers while hiring. However, we have observed that ratings are skewed towards higher values and do not provide valuable insights about freelancers' abilities to do a quality work. Therefore, we propose a multidimensional assessment framework which evaluates freelancers on several dimensions. The proposed framework, not only uses the current information about the freelancer, but also utilizes the past jobs he has performed. The framework is evaluated on the data collected from a popular online marketplace. Our analysis, performed on 7254 jobs and 96,271 applicants, shows that the assessment made by the proposed framework outperforms the baseline algorithm.