The lattice cryptosystem is considered to be able to resist the attacks of quantum computers. Lattice-based Public Key Encryption (PKE) schemes have attracted the interest of many researchers. In lattice-based cryptography, Learning With Errors (LWE) problem is a hard problem usually used to construct PKE scheme. To ensure the correctness of decryption, LWE-based schemes have a large ciphertext size. This makes these encryption schemes not practical enough when the communication bandwidth is limited. We propose a new variant of LWE, named Learning With Modulus (LWM) and prove that the new problem can be reduced from LWE problem. The proof idea of our reduction is similar to the reduction of LWR problem. We also construct a new PKE scheme based on the proposed LWM and LWE, which has small ciphertext size. For a 128 bits plaintext, the ciphertext size of our scheme is 53.57% of Lindner-Peikert's (LP) scheme under the same security level. We use python to test the performance of our scheme. The results show that our scheme is only about 0.015 ms slower than LP in the decryption. The performance of our scheme for generating keys and encrypting messages is similar to LP. The lattice cryptosystem is considered to be able to resist the attacks of quantum computers. Lattice-based Public Key Encryption (PKE) schemes have attracted the interest of many researchers. In lattice-based cryptography, Learning With Errors (LWE) problem is a hard problem usually used to construct PKE scheme. To ensure the correctness of decryption, LWE-based schemes have a large ciphertext size. This makes these encryption schemes not practical enough when the communication bandwidth is limited. We propose a new variant of LWE, named Learning With Modulus (LWM) and prove that the new problem can be reduced from LWE problem. The proof idea of our reduction is similar to the reduction of LWR problem. We also construct a new PKE scheme based on the proposed LWM and LWE, which has small ciphertext size. For a 128 bits plaintext, the ciphertext size of our scheme is 53.57% of Lindner-Peikert's (LP) scheme under the same security level. We use python to test the performance of our scheme. The results show that our scheme is only about 0.015 ms slower than LP in the decryption. The performance of our scheme for generating keys and encrypting messages is similar to LP. The lattice cryptosystem is considered to be able to resist the attacks of quantum computers. Lattice-based Public Key Encryption (PKE) schemes have attracted the interest of many researchers. In lattice-based cryptography, Learning With Errors (LWE) problem is a hard problem usually used to construct PKE scheme. To ensure the correctness of decryption, LWE-based schemes have a large ciphertext size. This makes these encryption schemes not practical enough when the communication bandwidth is limited. We propose a new variant of LWE, named Learning With Modulus (LWM) and prove that the new problem can be reduced from LWE problem. The proof idea of our reduction is similar to the reduction of LWR problem. We also construct a new PKE scheme based on the proposed LWM and LWE, which has small ciphertext size. For a 128 bits plaintext, the ciphertext size of our scheme is 53.57% of Lindner-Peikert's (LP) scheme under the same security level. We use python to test the performance of our scheme. The results show that our scheme is only about 0.015 ms slower than LP in the decryption. The performance of our scheme for generating keys and encrypting messages is similar to LP.