A comprehensive multimodal benchmark of neuromorphic training frameworks for spiking neural networks

Spiking neural networks (SNNs) represent a promising paradigm for energy-efficient, event-driven artificial intelligence, owing to their biological plausibility and unique temporal processing capabilities. Despite the rapid growth of neuromorphic training frameworks, the lack of standardized benchmarks hinders both the effective comparison of these tools and the broader advancement of SNN-based solutions for real-world applications. In this work, we address this critical gap by conducting a comprehensive, multimodal benchmark of five leading SNN frameworks-SpikingJelly, BrainCog, Sinabs, SNNGrow, and Lava. Our evaluation system integrates quantitative performance metrics-including accuracy, latency, energy consumption, and noise immunity-across diverse datasets (image, text, and neuromorphic event data), along with qualitative assessments of framework adaptability, model complexity, neuromorphic features, and community engagement. Our results indicate that SpikingJelly excels in overall performance, particularly in energy efficiency, while BrainCog demonstrates robust performance on complex tasks. Sinabs and SNNGrow offer balanced performance in latency and stability, though SNNGrow shows limitations in advanced training support and neuromorphic features, and Lava appears less adaptable to large-scale datasets. Additionally, we investigate the effects of varying time steps, training methods, and data encoding strategies on performance. This benchmark not only provides actionable guidance for selecting and optimizing SNN solutions but also lays the foundation for future research on advanced architectures and training techniques, ultimately accelerating the adoption of energy-efficient, brain-inspired computing in practical artificial intelligence engineering.