Recognizing Scoring in Basketball Game from AER Sequence by Spiking Neural Networks

The automatic score detection and recognition in basketball game has important application potentials, for examples, basketball technique analysis and 24 second control in the game. Although existing studies have been conducted on broadcast videos, most of them usually learned a machine learning algorithm on long videos recorded by traditional cameras. Address Event Representation (AER) sensor provides a possibility to deal with the problem by a human sensing manner. It represents the visual information as a series of spike-based events and records event sequences. Compared to traditional videos, AER events can fully utilize their addresses and timestamp information, forming precise spatio-temporal features with significantly less storage cost. More importantly, it issues spikes which can be naturally processed by human-style spiking neural networks (SNNs). In this paper, we propose to recognize scoring in basketball game from AER sequences. A new model is designed to extract dynamic features and discriminate different event streams using SNN. To handle the imbalance problem between positive and negative samples, we use an imbalanced Tempotron algorithm in our SNN model. Meanwhile, an AER sequence dataset of basketball games is collected. The experimental results demonstrate that our method achieves better performance compared with existing models.