Hardware-friendly Activation Functions for HybridViT Models

In recent years, CNN+ViT hybrid models have shown promising performance in computer vision tasks. To implement the CNN+ViT Hybrid model in resource-limited devices, various studies have been ongoing to address issues of parameter size and computational complexity through quantization, aiming to enable hardware-friendly low-bit integer operations. However, commonly used ViT activation functions (e.g., GeLU, Swish) inevitably require floating-point operations. To address this problem, some studies have been conducted to approximate these functions with alternatives that allow integer operations. Inspired by the Shift-GeLU approach, which approximates the GeLU function to enable integer operations, we propose and evaluate the Shift-Swish function on the MobileViT model at both software and hardware levels. Experimental results show that the hardware-level RTL design of the proposed method can reduce LUT by 63.25%, FF usage by 87.69%, and power consumption by 46.57% with a minimum accuracy drop of 0.6% compared to the baseline.