Abstract. Cloud segmentation plays a very important role in the astronomical observatory site selection. At present, few researchers segment cloud in the nocturnal All Sky Imager (ASI) images. This paper proposes a new automatic cloud segmentation algorithm which utilizes the advantages of deep learning fully convolutional networks (FCN) to segment cloud pixels from diurnal and nocturnal ASI images, named enhancement fully convolutional networks (EFCN). Firstly, all the ASI images in the data set from the Key Laboratory of Optical Astronomy at the National Astronomical Observatories of Chinese Academy of Sciences (CAS) are converted from the red-green-blue (RGB) color space to hue-saturation-intensity (HSI) color space. Secondly, the channel of the HSI color space is enhanced by the histogram equalization. Thirdly, all the ASI images are converted from the HSI color space to RGB color space. Then after 100000 times iterative training based on the ASI images in the training set, the optimum associated parameters of the EFCN-8s model are obtained. Finally, we use the trained EFCN-8s to segment the cloud pixels of the ASI image in the test set. In the experiments our proposed EFCN-8s was compared with other four algorithms (OTSU, FCN-8s, EFCN-32s, and EFCN-16s) using four evaluation metrics. Experiments show that the EFCN-8s is much more accurate in the cloud segmentation for diurnal and nocturnal ASI images than other four algorithms.