In Internet of Things (IoT), millions of devices communicate sensitive information with each other. Ensuring secure data generation and communication between the devices is critical and challenging. Automatic and secure authentication of users to access such critical services is a challenge. In this paper, we propose an ECG-based lightweight biometric-based authentication scheme based on Bio-CNN. The proposed scheme uses CNN architecture with suitable convolutional filters to properly analyze an ECG signal. We implement a hashing concept with the CNN features to make our system more secure. To examine the suitability of the proposed Bio-CNN-based scheme in an IoT device, we have tested all our experiments on a Raspberry Pi. Finally, the experiment results prove that the proposed method outperforms different CNN architectures and various state-of-the-art authentication techniques in terms of accuracy, precision, recall, etc., enhancing the overall performance.