Combination of directional modulation (DM) and frequency diverse array (FDA) provides a novel opportunity for enabling physical layer security because of the enhanced ability of distance resolution. However, in the existing studies, the time-varying nature of the FDA pattern is usually ignored. In this paper, a modified model of FDA-DM is proposed, in which the ignored time factor in the previous studies is taken into consideration for providing a new and more accurate perspective on evaluating the security of FDA-DM. Furthermore, we reveal that FDA-DM can achieve angle-dependent and directional-time-coupled-dependent security. After that, based on the retrodirective FDA (RFDA) with a new structure, a novel synthesis-free FDA-DM scheme is proposed for both overcoming the limitations of the FDA-DM scheme and allowing self-tracking the position of the pilot signal without any prior knowledge. Meanwhile, a new set of nonlinear frequency offsets, defined as rearranged linear frequency offsets (RLFOs), is also proposed for both combining the advantages of the linear/nonlinear frequency offsets and bringing conveniences to practical implementation. In addition, the closed-form expression of the average signal-to-artificial-noise-ratio (SANR) is given out for evaluating the security performance of the proposed scheme. Finally, numerical results are presented to verify both the accuracy of the proposed theoretical analysis and the superiority of the RFDA-DM scheme.