An optimization technique for computer-aided design of waveguide gratings is developed to obtain asymmetric reflection characteristics. The optimal grating pattern is constructed by modulating the phase of a dense periodic grating with a complex-valued function. The modulation function is quantized with respect to its phase and represented by a digital vector. The optimization is carried out by simulated annealing. An example of design is presented. Better results are obtained by selecting a larger number of quantized phase levels.