From a study of interaction of signal and pump oscillations, an explanation for instabilities and a high noise level observed in Josephson junction parametric amplifiers is presented. It is shown that these drawbacks of the amplifiers are due to the weakly nonlinear chaotic states, that appears to be typical for such devices. The conditions for, and the mechanisms of, the transition to chaos are discussed.