This article proposes an edge-based denoising algorithm to restore the original image, which is highly degraded by the salt and pepper noise. Most of the existing image denoising algorithms consider edge as a noise. Here, the proposed algorithm can set out to resolve this ambiguity. The concept of directional filters is being used to delineate the edges from noise. The proposed algorithm performance is tested for different noise densities ranging from 5% to 90% on both the greyscale and colour images. It is compared with the current state of art techniques using several performance metrics such as peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) values, and image enhancement factor (IEF). The results showed that the proposed algorithm has achieved an improvement of 60% over the state of art techniques.