We propose a PDE-based image inpainting method using anisotropic heat transfer model, which can simultaneously propagate the structure and texture information. In structure inpainting, the propagating direction and intensity are related to image contents, and the strength of propagation along gradient direction is made inversely proportional to the magnitude of gradient. In texture inpainting, the added texture term reflects periodicity along the texture and its perpendicular direction. For numerical implementation, the step size of finite difference is adaptively chosen according to the curvature, leading to fewer iteration steps and satisfactory inpainting quality. Compared with other high order PDE methods and layered methods, the proposed approach is more concise and doesn't need image decomposition. Experiments are carried out to show effectiveness of the method.