Facial Expression Editing in Face Sketch using Shape Space Theory

Facial expression editing in face sketch is an important and challenging problem in computer vision community as facial animation and modeling. For criminal investigation and portrait drawing, automatic expression editing tools for face sketch improve work efficiency obviously and reduce professional requirements for users. In this paper, we propose a novel method for facial expression editing in face sketch using shape space theory. The new facial expressions in the sketch images can be regenerated automatically. The method includes two components: 1) face sketch modeling; 2) expression editing. The face sketch modeling constructs 3D face sketch data from 3D facial database to match the 2D face sketch. Using facial landmarks, the "shape" of the face sketch is represented in shape space. The shape space is a manifold space which removes the rigid transform group. In shape space, the accurate 3D face sketch model is obtained which is consistent to the original 2D face sketch. For expression editing, we change the parameters of 3D face sketch model in the shape space to obtain new expressions. The expression transfer in 3D face sketch model can be mapped into the 2D face sketch. The advantages of our method are: full-automatic in modeling process; no requirements of drawing skills to user and friendly interaction; robustness to head poses and different scales. In experiments, we use the 3D facial database, FaceWareHouse, to construct the 3D face sketch model and use face sketch images from database: CUHK Face sketch Database (CUFS) to show the performance of expression editing. Experimental results demonstrate that our method can effectively edit facial expressions in face sketch with high consistency and fidelity.