The stability analysis of a genetic algorithm-based adaptive neural network controller for a nonlinear system is discussed. First, we track the reference trajectory for an uncertain and nonlinear plant. We make sure that it is well approximated and described using the structure of a radial based function network. Then, we decide on the initial values of the consequent parameter vector utilizing a genetic algorithm. Next, a modified adaptive neural network controller is proposed to simultaneously stabilize and control the system. A stability criterion is also derived from Lyapunov's direct method to ensure the stability of the nonlinear system. Finally, we discuss an example and provide a numerical simulation. The results demonstrate that the control methodology can rapidly and efficiently control a complex and nonlinear system.