Millimeter-wave (mmWave) systems must overcome severe phase noise impairment to exploit its full advantage (e.g. high data rate). However, the estimation of phase noise combined with wireless channel requires not only high-computational complexity, but also does not guarantee a stable solution. In this paper, the fact that the coherence bandwidth of a mmWave system is much larger than those of conventional systems is exploited to acquire the knowledge regarding phase noise by linear estimator. The channel coefficients are assumed to be piecewise-constant over Ncb successive subcarriers within coherence bandwidth. Utilizing the coherence structure, we propose a new algorithm for phase noise compensation on orthogonal frequency division multiplexing (OFDM) systems. Numerical results with phase noise generated as a Wiener process demonstrate that the proposed algorithm can provide improved performance even in severe phase noise environments.