Aesthetics is concerned with the beauty and art of things in the world. Judging the aesthetics of images is a highly subjective task. Recently, deep learning-based approaches have achieved great success in image aesthetic assessment problem. In this paper, we have implemented various multi-channel Convolution Neural Network (CNN) architectures to classify images in high and low aesthetic quality. Class activation maps of images are used as input to one channel along with variation of raw images in the proposed two-channel deep network architecture. Various pre-trained deep learning models such as VGG19, InceptionV3, Resnet50 have been implemented in the proposed multi-channel CNN architecture. Experiments are reported on the AVA dataset, which shows improvement in the image aesthetic assessment task over existing approaches.