How to ensure the security and efficiency of real-time video transmission in the existing network environment is increasingly becoming an urgent problem to be solved. This paper presents a moving object-based video encryption algorithm, which can satisfactorily resolve the tradeoffs between security and efficiency of real-time video transmission. A real-time video encryption algorithm based on moving objects is proposed, which combines the extraction of moving objects with the entropy coding-based encryption algorithm, and encrypt only the extracted moving objects. Experimental results show that the algorithm has small computational overhead, and good encryption real-timeliness, which can achieve favourable tradeoffs between security and coding efficiency, and meet real-timeliness requirements.