This study aims at improving the efficiency of remote sensing scene classification (RSSC) through lightweight neural networks and to provide a possibility for large-scale, intelligent and real-time computation in performing RSSC for common devices. In this study, a lightweight RSSC model is proposed, which is named RSCNet. First, we use the lightweight ShuffleNet v2 network to extract the abstract features from the images, which can guarantee the efficiency of the model. Then, the weights of the backbone are initialized using transfer learning, allowing the model to learn by drawing on the knowledge of ImageNet. Second, to further improve the classification accuracy of the model, we propose to combine ShuffleNet v2 with an efficient channel attention mechanism that allows the features of the input classifier to be weighted. Third, we use a regularization technique during the training process, which utilizes label smoothing regularization to replace the original loss function. The experimental results show that the classification accuracy of RSCNet is 96.75% and 99.05% on the AID and UCMerced_LandUse datasets, respectively. The floating-point operations (FLOPs) of the proposed model are only 153.71 M, and the time spent for a single inference on the CPU is about 2.75 ms. Compared with existing RSSC methods, RSCNet achieves relatively high accuracy at a very small computational cost.