In the practice of warm asphalt mixture (WMA), many technologies can be employed to lower the viscosity of asphalt, thereby reducing the mixing temperature by 20-30 degrees C or more. It is well-received for the merits of construction convenience, low energy consumption, and less aging of mixtures. Existing research has found that specific warm-mixing technology and relatively-low mixing temperature may cause water residue in the mixture, further depressing the performance, especially moisture susceptibility. In order to analyze the mechanism of this issue, the surface free energy (SFE) and corresponding laboratory test are applied to research the factors affecting moisture susceptibility systematically. First, this study puts forward a new indicator based on SFE, namely effective adhesion work, to evaluate the adhesion property between asphalt and aggregates with different moisture contents. Referencing the existing two-phase model of asphalt-aggregate, the three-phase model of asphalt-aggregate-warm mixing additives is proposed to simulate the mixing process of dry mixing method, in which warm mixing additives is added separately. Then combine the existing and newly-proposed models and indicators to analyze the influence of aggregate type, aggregate moisture content, warm mixing additives type, and mixing process to the moisture susceptibility of WMA. Finally, the freeze-thaw split test is carried out to verify the effectiveness of the indicators based on SFE characterizing the moisture susceptibility. The results show that those above factors could affect the interfacial adhesion, further affecting the moisture susceptibility of WMA, among which the moisture content is particularly prominent. At the same time, it is proved that the effective adhesion work and three-phase model proposed in this paper can be effectively used in the evaluation of the adhesion among aggregate, asphalt and warm mixing additives.