“…Despite the success in most vision tasks, ViT-based models cannot compete with the well-studied lightweight CNNs [21,49] when the inference speed is the major concern [50,51,52], especially on resource-constrained edge devices [17]. To accelerate ViT, many approaches have been introduced with different methodologies, such as proposing new architectures or modules [53,54,55,56,57,58], re-thinking self-attention and sparse-attention mechanisms [59,60,61,62,63,64,65], and utilizing search algorithms that are widely explored in CNNs to find smaller and faster ViTs [66,28,29,67]. Recently, LeViT [23] proposes a CONV-clothing design to accelerate vision transformer.…”