The millimeter-wave (mmWave) communications is a promising technology for next-generation wireless networks with its available broad spectrum. Along with massive number of antennas employed at the transmitter and the receiver, the number of unknown channel coefficients become extremely large. Exploiting the sparse nature of mmWave channels, this paper proposes a parameter perturbation based sparse recovery technique for mmWave channel estimation. Recently, classical compressive sensing (CS) based sparse recovery techniques have been applied in this area. However, CS based reconstructions are highly affected by basis mismatch problems such as off-the-grid targets, or equivalently, scattering points. The proposed iterative algorithm called parameter perturbed orthogonal matching pursuit (PP-OMP) jointly solves for both the sparse signal, which is the unknown mmWave channel itself, and the basis mismatch due to off-the-grid problem. We verify through extensive numerical results that the proposed PP-OMP algorithm achieves significantly better channel estimation performance compared to the state of the art sparse reconstruction techniques.