With the development of information technology, more and more web services have emerged, thereby making it difficult for customers to find their favorite services quickly and accurately. To overcome this difficulty, recently the collaborative filtering (CF) technique has been widely employed for personalized service recommendation, meanwhile improving the profits of service providers. Although the CF-based web service recommender systems have shown their potential, they appear to be vulnerable to shilling attack problems. Therefore, in this paper we analyze a general form of web service shilling attacks and four kinds of classical attack models, e.g., average attack, bandwagon attack, random attack, and segment attack are thoroughly investigated. Furthermore, we also study the impact of distributionaware Pareto attack models. To demonstrate how shilling attacks alter the recommendation results, this paper analyzes 1) the variation of Quality-of-Service (QoS) prediction values of target services, 2) the QoS value prediction shifts of services with short response time which are more likely recommended, and 3) the comparison of prediction shift caused by classical attack models and Pareto attack models. The experimental results on WS-DREAM dataset revealed several interesting findings about the predictions of QoS values of target service correlated to different attack models. It is expected that this work can provide some insight for future vulnerability analysis of CF-based web service recommender systems.