Protecting transmission lines frequently involves applying distance relays. Protective relays must filter their inputs to reject unwanted quantities and retain signal quantities of relevant interest. Accuracy and convergent speed of filter algorithm are essential for protective relays. A widely applied filter algorithm, the Discrete Fourier Transform (DFT) can easily remove integer harmonics using simple calculation. However, the voltage and current signals contain serious harmonics and decaying de during the fault interval. In addition, the decaying de and higher order harmonics seriously decrease the precision and convergence speed of fundamental frequency signal from DFT. In this investigation, we derive a novel algorithm which combines the appropriate analog low pass filter and modified Full Cycle DFT (FCDFT) or Half Cycle DFT (HCDFT) algorithm to remove the decaying de in a voltage or current signal, Using the Electromagnetic Transients Program (EMTP) simulates the transient responses of transmission lines during the fault period. Applying the proposed algorithm in distance relays effectively suppresses the decaying de and quickly decomposes the accurate fundamental frequency components.