This paper proposes a fast and unified encryption and decryption algorithm based on a composite chaotic system. By combining Logistic map and Sine map, the New-Logistic-Sine map (NLS map) is obtained. NLS map generated the diffusion key matrix needed in the algorithm process, which can enhance the anti-attack ability of the encryption algorithm. Different from most image cryptography systems, the algorithm adopted in this paper has the same encryption process and decryption process, which can save half of the resources in real applications. Firstly, the Secure Hash Algorithm 256 (SHA256) value of the original image was obtained, and the initial values and control parameters of NLS map and Logistic map were calculated; Secondly, the diffusion key matrix is obtained by iterative the NLS map, and is used to perform the first diffusion of the original image; Thirdly, the permutation key sequence is obtained by iterative the Logistic map, and using the sequence to perform the permutation operation on the image after the first diffusion; Finally, the same diffusion key matrix as the first diffusion operation is used to carry out the second diffusion operation on the displaced image to obtain the final encrypted image. The simulation experiment and security analysis show that the proposed image cryptosystem possessed identical encryption process and decryption process, and the algorithm speed is improved ensure the security of the algorithm.