Cloud manufacturing (CMfg) is a new service-oriented smart manufacturing paradigm, and it provides a new product development model in which users are enabled to configure, select, and utilize customized manufacturing service on-demand. Because of the massive manufacturing resources, various users with individualized demands, heterogeneous manufacturing system or platform, and different data type or file type, CMfg is fully recognized as a kind of complex manufacturing system in complex environment and has received considerable attention in recent years. In practical scenarios of CMfg, the amount of manufacturing task may be very large, and there are always quite a lot of candidate manufacturing services in cloud pool for corresponding subtasks. These candidate services will be selected and composed together to complete a complex manufacturing task. Obviously, manufacturing service composition plays a very important role in CMfg lifecycle and thus enables complex manufacturing system to be stable, safe, reliable, and efficient and effective. In this paper, a new manufacturing service composition scheme named as Multi-Batch Subtasks Parallel-Hybrid Execution Cloud Service Composition for Cloud Manufacturing (MBSPHE-CSCCM) is proposed, and such composition is one of the most difficult combination optimization problems with NP-hard complexity. To address the problem, a novel optimization method named as Improved Hybrid Differential Evolution and Teaching Based Optimization (IHDETBO) is proposed and introduced in detail. The results obtained by simulation experiments and case study validate the effectiveness and feasibility of the proposed algorithm.