Stochastic Neural Networks with Layer-Wise Adjustable Sequence Length

The implementation of Neural Networks (NNs) on resource-limited devices poses significant challenges, with Stochastic Computing (SC) as a solution for efficient execution. By representing values as sequences of bits that are processed serially, SC implementations significantly reduce the energy dissipation of NNs. However, with NNs growing in size, SC can also become energy-intensive, prompting a need for enhanced efficiency. This paper introduces Adjustable Sequence Length (ASL), a method employing varied sequence lengths across different NN layers to reduce energy/latency overheads with negligible impact on performance. The feasibility of sequence truncation across layers is assessed; as per simulation results, the ASL method demonstrates significant savings in energy of up to 40.41% and the savings in latency of up to 54.74% when compared with conventional SC implementations.