In order to extract characteristics of face by making full use of LBP and improve its "adaptive ability", we proposed an algorithm based on global and local fusion LBP. First, we will extract overall face feature histogram with LBP, then segment the image into blocks, extract each LBP histogram feature, then combine the global and local features according to certain order, and we regard it as the total character of image. Finally, we use the LS- SVM (least squares support vector machine) identifying and training samples of face image to improve the speed of recognition the experiment on ORL face database shows that the algorithm has high recognition rate and has improved recognition rate of the face.