Non-Gaussian elliptically contoured autoregressive moving average (EC-ARMA) processes are defined as ARMA models with non-Gaussian noise. The driving noise series is uncorrelated but generally dependent, and follows elliptically contoured distributions. Moments, characteristic functions, parametric representations, and other probabilistic characteristics are given for elliptically contoured vectors and EC-ARMA processes. These characteristics are used to outline methods for generating samples of EC-ARMA processes. The analysis includes classical and degenerated elliptically contoured driving noises. The Cauchy noise is an example of a degenerated noise, in the sense that it does not have moments. It is shown that EC-ARMA models with finite first two moments become Gaussian as time increases. However, degenerate EC-ARMA models remain non-Gaussian at any time. Several numerical examples are used to demonstrate features of EC-ARMA processes and methods to generate realizations of these processes.