Shadow Detection and removal from images is a challenging task in visual surveillance and computer vision applications. The appearance of shadows creates severe problems. There are various methods already exists but scope in this area is wide and open. In this paper, Optimization of Shadow Detection and Removal using Improved Artificial Bee Colony Algorithm (IABC) is proposed. The proposed method uses edge map, multilevel thresholds, masking, boundaries evaluation and, IABC algorithm. First data pre-processing is applied to find the correlation between the pixels then three level low, medium and high value of thresholds and the corresponding value of masking and boundaries are calculated to accurately differentiate pixels as foreground. The edge response, curvature, gradient are applied to find the true location of boundaries. Finally, IABC has been applied for detecting the shadow and median filter is used to remove the shadow. The results show improvement as compared to other existing methods