In prefabricated buildings, the building walls act as thermal insulation layers of the building envelope. The walls in the envelope consume 50% of the energy consumed by the whole building. To promote prefabricated buildings, it is very important to design energy-saving exterior walls. Focusing on prefabricated building in Inner Mongolia, this paper designs a novel composite thermal insulation block for building walls based on coal gangue ceramsites and polyethylene insulation boards, and probes deep into its mechanical properties and environmental thermal performance. The results show that, with the growing content of coal gangue ceramsites, the composite thermal insulation block saw a linear decline in volume density, compressive strength, and thermal conductivity; the theoretical heat transfer coefficient of the proposed insulation block deviated slightly from the calculated value, indicating the feasibility of the calculated heat transfer coefficient. The research results promote the application of new building walls in the design of prefabricated buildings.