Cloudmaps from static ground-view video

Abstract

Cloud shadows dramatically affect the appearance of outdoor scenes. We describe three approaches that use video of cloud shadows to estimate a cloudmap, a spatio-temporal function that represents the clouds passing over the scene. Two of the methods make assumptions about the camera and/or scene geometry. The third method uses techniques from manifold learning and does not require such assumptions. None of the methods require directly viewing the clouds, but instead use the pattern of intensity changes caused by the cloud shadows. An accurate estimate of the cloudmap has potential applications in solar power estimation and forecasting, surveillance, and graphics. We present a quantitative evaluation of our methods on synthetic scenes and show qualitative results on real scenes. We also demonstrate the use of a cloudmap for foreground object detection and video editing.

A Unified Model for Near and Remote Sensing

“…A number of papers have tried to estimate properties of weather from geotagged and timestamped ground-level imagery. For example, Murdock et al [21,22] and Jacobs et al [11] use webcams to infer cloud cover maps, Li et al [16] use ground-level photos to estimate smog conditions, Glasner et al [8] estimate temperature, Zhou et al [37] and Lee et al [14] estimate demographic properties, Fedorov et al [5,6] and Wang et al [27] infer snow cover, Khosla et al [12] and Porzi et al [23] measure perceived crime levels, Leung and Newsam [15] estimate land use, and so on.…”

Convective-Scale Assimilation of Cloud Cover from Photographs Using a Machine Learning Forward Operator

“…A number of authors use methods of artificial intelligence to classify the scenes seen by the cameras in terms of weather conditions. This specifically relates to the detection of cloudiness conditions for applications in scene recognition, traffic monitoring, surveillance and computer graphics (Jacobs et al 2016;Islam et al 2013;Lu et al 2014). Additionally to cloudiness, some authors also classify webcam images regarding rain or snow occurrence, as well as visibility parameters such as fog or haze (Chu et al 2017;Zhang et al 2016;Moodley and Viriri 2018).…”