In this paper, the cascaded alpha - mu fading distribution is first introduced and mathematically characterized, which arises as a generalization of the cascaded Rayleigh, Weibull, and Nakagami-m fading distribution, by properly selecting fading parameters a and mu with specific values. In particular, the statistical characterization of the cascaded alpha - mu fading channels, namely, the probability density function and cumulative distribution function, are first studied. This set of new statistical results is applied to the modeling and analysis of the reliability and security performance of wireless communication systems over the cascaded alpha - mu fading channel. Regarding system reliability, the amount of fading, outage probability, average channel capacity, and the average symbol error probability with coherent and non-coherent demodulation schemes are derived with respect to the univariate Fox's H-function. In terms of security analysis, the secrecy outage probability P-out, the probability of non-zero secrecy capacity P-nz, and the average secrecy capacity are analyzed in the exact closed-form expressions which are derived in the presence of an active eavesdropper. In addition, an asymptotic analysis of all aforementioned metrics is carried out, in order to gain more insights of the effect of the key system parameters on the reliability and security. Tractable results are computed in terms of the Fox's H-function and later on are successfully validated through Monte-Carlo simulations.