Automatic Operator Performance Tuning in a Machine Learning System on Edge

With the current large scale deployment of machine learning technologies, such as those on cloud servers and edge and IoT hardwares, machine learning systems have been widely prevalence. Practical requirement has driven their performance increase in both academia and industry. However, the application requirement varies greatly across different applications, and directly using off-the-shelf systems might not be sufficient in many cases. In this work, we first propose to implement a series of techniques to optimize performance of convolution operation, one of the most important operations, in constructing deep learning networks. Besides, we also propose to apply the automated empirical optimisation of software approach to improve the performance of operators in machine learning system, most notably across various hardware platforms. Evaluation compared to existing libraries on different hardware devices has proved the efficiency of our proposed method.