Image denoising is a fundamental tool in the fields of image processing and computer vision. With the rapid development of multimedia and cloud computing, it has become popular for resource-constrained users to outsource the storage and denoising of massive images. However, it may cause privacy concerns and response delays. In this scenario, we propose an efFicient privAcy-preseRving Image deNoising schEme (FARINE) for outsourcing digital images. By introducing a key conversion mechanism, FARINE allows removing noise from a given noisy image using a non-local mean way without leaking any information about the plaintext content. Due to its low computational latency/communication cost, edge computing is considered to improve the user experience. To achieve a dynamic user set efficiently, we design a fine-grained access control mechanism to support user authorization and revocation in multi-user scenarios. Extensive experiments over several benchmark data sets show that FARINE obtains comparable performance to plaintext image denoising.