Nowadays a lot of effort is spent on developing OFDM-based inexpensive wireless transceivers. Direct-conversion radio frequency transceivers are appealing because they avoid costly IF analog components. This kind of transceivers imply analog RF I/Q separation. The mismatch between the analog components in the in-phase and quadrature branches introduces an unwanted in-band interference. Unfortunately, OFDM-based systems are very sensitive to I/Q mismatch, mostly when high order modulation schemes are applied. A digital compensation of this unwanted effect is required. In this paper, we developed an iterative method for estimating and compensating the transmitter/receiver frequency-dependent I/Q imbalance jointly with the propagation channel in the frequency domain. The new estimation technique resorts to the iterative Expectation-Maximization algorithm in order converge to the ML estimation for both channel and IQ imbalance coefficients in presence of unknown transmitted data symbols. Although the the estimation may be perform in the absence of known pilot symbols, the algorithm provides flexibility in terms of pilot sequence, allocation and load in order to improve its convergence. An advanced low complexity equalizer is proposed which compensates for the frequency-dependent IQ imbalance. Simulation results show optimal performance even in the absence of any pilot symbol and large IQ mismatches. The proposed technique enables the system to achieve high SNRs.