In recent years, we have been witnessing a dramatic increase on the personal loan for consumption, due to the rapid development of e-services, including e-commerce, e-finance and mobile payments. Resulting from the lack of effective grid verification and supervision, it inevitably leads to large-scale losses caused by credit loan fraud [1]. Considering the difficulty of manual inspection and verification on the large amount of credit card transactions, machine learning methods are commonly used to detect fraudulent transactions automatically. This article has applied the Extreme Gradient Boosting(XGBoost) model for data mining and analysis, which is inspired by its brilliant reputation in various data mining contests. With people's growing concern about privacy protection, how can we apply data mining techniques while taking consideration into privacy terms is one problem. Additionally, according to current loan fraud detection studies, some features are considered to contain little information or a bit of redundancy, whereas others hold the critical information which makes things harder when feature engineering. In order to filter useless information and preserve the useful information without knowing the meaning of our data, this paper combines Kernel Principal Component Analysis (Kernel PCA) together with XGBoost algorithm and proposes a new hybrid unsupervised and supervised learning model, KP-XGBoost. We use grid search to avoid over-fitting and compare the performance of both XGBoost and P-XGBoost and other classical machine learning methods. It turns out that P-XGBoost outperforms XGBoost in fraud detection, which provides a new perspective to detecting the fraud behaviour while protecting clients' privacy.