Learning Deterministic Spiking Neuron Feedback Controllers

We consider the problem of feedback control when the controller is constructed solely of deterministic spiking neurons. Although spiking neurons and networks have been the subject of several previous studies, analysis has primarily been restricted to a firing rate model. In contrast, we construct a deterministic spiking neuron controller whose control output is one or multiple sparse spike trains. We model the problem formally as a hybrid dynamical system comprised of a closed loop between a plant and a spiking neuron network controller. The construction differs from classical controllers owing to the fact that the control feedback to the plant is generated by convolving the spike trains with a fixed kernel, resulting in a highly constrained and stereotyped control signal. We derive a novel synaptic weight update rule via which the spiking neuron controller learns to hold process variables at desired set points. We demonstrate the efficacy of the rule by applying it to the classical problem of the cart-pole (inverted pendulum). Experiments demonstrate that the proposed controller has a larger region of stability as compared to the traditional PID controller and its trajectories differ from those of the PID controller.