In this paper, we consider a Van der Pol-Duffing oscillator that is excited parametrically by a small intensity real noise, which is assumed to be an integrable function of an n-dimensional Ornstein-Uhlenbeck vector process that is an output of a linear filter system. The stability properties include the moment Lyapunov exponent g(p, x(0)) and the maximal Lyapunov exponent, and the stability in probability are examined. To study a model of enhanced generality, we remove both the detailed balance condition and the strong mixing condition. In the case of an arbitrary finite real number p, we employ the perturbation method and a spectrum representation of the Fokker-Planck operator of the linear filter system to construct asymptotic expansions of the pth moment Lyapunov exponent and the top Lyapunov exponent. The same methods are also used for a nonlinear stochastic system to obtain the FPK (Fokker-Planck-Kolmogonov) equation for the amplitude process, which is identical to the one that is derived from the stochastic averaging method in the case of a broadband noise excitation. On the basis of this FPK equation, we also examine the almost-sure stability condition of the Ito stochastic differential equation for the amplitude process, which matches the result that is derived from the maximal Lyapunov exponent. Finally, the method proposed by Lin and Cai (Probabilistic Structural Dynamics, Advanced Theory and Application, McGraw-Hill, New York, 1995) is adopted to examine the stability in probability of the amplitude process for the nonlinear Ito differential equation. (c) 2004 Elsevier Ltd. All rights reserved.